Large model acceleration system and method based on random calculation, medium, terminal and program product

Abstract

The invention provides a large model acceleration system and method based on random calculation, a medium, a terminal and a program product, and the system comprises a random bit stream coding module which is used for coding a weight parameter and an input feature of a large model into a random bit stream with a preset bit number from a floating-point number; the random calculation acceleration module is used for carrying out multiply-add operation and activation function operation according to the weight parameters converted into the random bit streams and the input characteristics so as to obtain an operation result; and the random bit stream decoding module is used for decoding the operation result of the random bit stream into a floating-point number format. According to the method, on the premise of ensuring the reasoning and training precision of the large model, the hardware cost is reduced, the computing architecture is simplified, the energy efficiency ratio is improved, meanwhile, the mainstream large model is supported, and the universality and deployment flexibility are considered.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a large model acceleration system, method, medium, terminal and program product based on random computation. Background Technology

[0002] Currently, pre-trained large models based on the Transformer architecture (such as the GPT series, LLaMA series, BERT, etc.) are widely used in natural language processing, generative AI, computer vision, and other fields. However, the high computing power requirements of large models lead to high deployment costs for their acceleration. To address this challenge, existing acceleration solutions mainly focus on two directions: hardware and software. At the hardware level, they primarily rely on high-performance dedicated computing chips, such as NVIDIA H100 GPUs, Google TPU v5e, and Cambricon MSI 370 ASIC chips. These chips improve computing density through customized computing units (such as Tensor Cores in GPUs). Meanwhile, some scenarios use FPGAs (such as the Xilinx Zynq UltraScale+ series) to achieve flexible acceleration. However, traditional FPGA designs are based on binary computing architectures, and the utilization rate of their hardware resources (such as LUT lookup tables and registers) is only 30%-50%, making it difficult to further reduce costs. At the software level, computational overhead is reduced through algorithm optimization, specifically including: quantization techniques, which compress model parameters from FP32 / FP16 to INT8 / INT4, with typical algorithms such as GPTQ and AWQ used to reduce storage and computation, but requiring additional calibration processes; pruning and distillation techniques, which remove redundant weights (e.g., pruning 50%-70% of low-contribution weights) or train small models to simulate the behavior of large models (e.g., MobileBERT is a distilled version of BERT); and parallel computing techniques, which break down tasks through data parallelism, model parallelism, and pipeline parallelism (e.g., the Megatron-LM framework), and utilize multi-GPU collaboration to improve speed, but require high-speed interconnects (e.g., NVLink), further increasing hardware investment.

[0003] While existing large-scale model acceleration solutions have improved computing power to some extent, they still suffer from the core drawback of balancing cost and performance, which is mainly manifested in the following ways:

[0004] (1) Excessive hardware costs: Dedicated chips such as GPUs / TPUs are expensive. The hardware investment for a large model inference cluster with 100 cards is 4 million to 6 million US dollars. In contrast, the R&D cycle of ASIC chips is as long as 1 to 2 years and the R&D cost exceeds 10 million US dollars. Moreover, the customized design is only compatible with a single model, has poor versatility, and cannot meet the needs of multi-model deployment.

[0005] (2) Low energy efficiency ratio leads to high long-term costs: Traditional binary computing chips have extremely high power consumption (such as the NVIDIA A10 GPU with a full load power consumption of 150W), and the annual electricity cost of a 100-card cluster exceeds US$130,000, resulting in significant long-term usage costs.

[0006] (3) Conflict between accuracy and acceleration effect: Although low bit quantization (such as INT2) can reduce the amount of computation, the accuracy loss can reach 5%-10%, which cannot meet the high-precision scenarios such as medical and financial fields. Furthermore, high pruning rate (such as pruning 70%) will lead to a decrease in model convergence and poor stability of inference results.

[0007] (4) High deployment threshold: Parallel computing requires deep optimization of hardware interconnection and software framework. Small and medium-sized enterprises lack professional technical teams and cannot afford the technical and cost investment in accelerating large models, which limits the widespread application of large models.

[0008] Therefore, it is necessary to provide a large-scale model acceleration system, method, medium, terminal, and program product based on random computation to solve the above-mentioned problems in the prior art. Summary of the Invention

[0009] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a large model acceleration system, method, medium, terminal and program product based on random computing, to solve the technical problems of high hardware cost, low energy efficiency, large accuracy loss and poor versatility of the prior art.

[0010] To achieve the above and other related objectives, the first aspect of this application provides a large model acceleration system based on random computation, comprising: a random bitstream encoding module for encoding the weight parameters and input features of the large model from floating-point numbers into a random bitstream of a preset number of bits; a random computation acceleration module for performing multiplication-addition operations and activation function operations based on the weight parameters and input features converted into random bitstreams to obtain the computation result; and a random bitstream decoding module for decoding the computation result of the random bitstream into floating-point format.

[0011] In some embodiments of the first aspect of this application, the random bitstream encoding module includes: a normalization unit, used to normalize the weight parameters and input features of a large model to a normalization interval and save the normalization coefficients; a pseudo-random number generation unit, used to generate uniform random numbers based on a configured feedback function; and an encoding unit, used to output a random bitstream of a preset number of bits by comparing the uniform random numbers generated by the pseudo-random number generation unit with the normalization interval of 0 to 1, and to encode the final random bitstream based on the uniform random numbers generated by the pseudo-random number generation unit using a bipolar encoding mechanism if the normalization interval is -1 to 1.

[0012] In some embodiments of the first aspect of this application, the process of encoding the uniform random number generated by the pseudo-random number generation unit based on the bipolar coding mechanism to obtain the final random bit stream specifically includes: encoding based on the input positive value and the uniform random number to output a random bit stream of a preset number of bits; calculating the positive part probability and the negative part probability based on the input negative number, and comparing the positive part probability, the negative part probability and the uniform random number to generate a positive random bit stream and a negative random bit stream of a preset number of bits respectively; subtracting the negative random bit stream from the generated positive random bit stream to obtain the final random bit stream.

[0013] In some embodiments of the first aspect of this application, the random computation acceleration module includes: a multiplication unit, used to input weight parameters and input features encoded as random bit streams into two input AND gates for multiplication to obtain the multiplication result of the random bit stream; an addition unit, used to input the multiplication result of the random bit stream output by the multiplication unit into an OR gate array for operation and output the accumulated result; a ReLU function calculation unit, used to compare the input original random bit stream with the all-zero random bit stream based on a comparator, and output the all-zero random bit stream if the input value is less than or equal to zero; and output the original random bit stream if the input value is greater than zero; and a GELU function calculation unit, used to input the input original random bit stream into an AND gate and an OR gate group for calculating the GELU function.

[0014] In some embodiments of the first aspect of this application, the random bitstream decoding module includes: a statistics unit, used to count the number of ones in the output random bitstream of a preset number of bits based on a counter; a probability calculation unit, used to calculate a probability value based on the count and the preset number of bits; and an inverse normalization unit, used to multiply the probability value output by the probability calculation unit by a normalization coefficient to obtain the final value in floating-point format.

[0015] In some embodiments of the first aspect of this application, the random bitstream decoding module further includes a calibration unit for calibrating the accumulated result output by the OR gate array by looking up a calibration table based on the calculated probability value, so as to obtain the true accumulated result.

[0016] To achieve the above and other related objectives, a second aspect of this application provides a method for accelerating large models based on random computation, comprising: encoding the weight parameters and input features of the large model into a random bit stream of a preset number of bits using floating-point numbers; performing multiplication-addition operations and activation function operations based on the weight parameters and input features converted to the random bit stream to obtain the computation result; and decoding the computation result of the random bit stream into a floating-point number format.

[0017] To achieve the above and other related objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method.

[0018] To achieve the above and other related objectives, a fourth aspect of this application provides a computer program product comprising computer program code that, when executed on a computer, causes the computer to implement the method.

[0019] To achieve the above and other related objectives, a fifth aspect of this application provides an electronic terminal, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the method.

[0020] As described above, the large model acceleration system, method, medium, terminal, and program products based on random computation of this application have the following beneficial effects:

[0021] This application uses a random bitstream encoding module to convert the weight parameters and input features of a large model from floating-point format to a random bitstream of a preset bit length. The converted weight parameters and input features are then input to a random computation acceleration module for multiplication, addition, and activation function operations to obtain the computation result. Finally, the computation result is input to a random bitstream decoding module for decoding to obtain the final computation result in floating-point format. This application reduces hardware costs, simplifies the computational architecture, and improves energy efficiency while ensuring the inference and training accuracy of large models. It also supports mainstream large models and balances versatility and deployment flexibility. Attached Figure Description

[0022] Figure 1 The diagram shown is a block diagram of a large model acceleration system based on random computation in one embodiment of this application.

[0023] Figure 2 The diagram shown is a schematic representation of the working principle of a large model acceleration system based on random computation in one embodiment of this application.

[0024] Figure 3 The diagram shown is a flowchart of a large model acceleration method based on random computation in one embodiment of this application.

[0025] Figure 4 The diagram shown is a structural schematic of an electronic terminal according to an embodiment of this application. Detailed Implementation

[0026] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0027] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. For example, "first XX" and "second XX" are merely used to distinguish different XXs and do not limit their order. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply that they are different.

[0028] It should be noted that, in the embodiments of this application, the words "exemplary" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0029] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0030] Before providing a further detailed description of the present invention, the nouns and terms used in the embodiments of the present invention are explained, and the nouns and terms used in the embodiments of the present invention are subject to the following interpretations:

[0031] <1> Stochastic computing (SC) is a novel computing paradigm that replaces deterministic binary values ​​with probabilistic signals. Its core lies in mapping deterministic values ​​to a probabilistic bit stream, performing arithmetic operations through statistical properties rather than precise bit weights. Compared to traditional deterministic numerical computing, stochastic computing has advantages such as low hardware overhead, high asymptotic accuracy, and high fault tolerance, and is widely used in digital signal processing, neural network acceleration, and edge computing.

[0032] <2> Python NumPy scripts are an open-source numerical computation extension library for efficiently processing large multidimensional arrays and matrices, while providing a large library of mathematical functions for manipulating these arrays.

[0033] <3> The ReLU (Rectified Linear Unit) function is a widely used activation function in artificial neural networks. It usually represents the ramp function in mathematics, defined as f(x) = max(0, x). In neural networks, the ReLU function serves as the activation function of neurons to provide nonlinear output results after linear transformation.

[0034] <4> GELU (Gaussian Error Linear Unit) function: a non-linear activation function widely used in deep learning models, especially in natural language processing tasks, such as BERT and GPT.

[0035] <5> Linear Feedback Shift Register (LFSR): A hardware device used to generate pseudo-random numbers. It works by performing an XOR operation on certain bits in the shift register to achieve feedback. An LFSR consists of multiple flip-flops and XOR gates. The initial value (random seed) and the taps (bits involved in the operation) determine the generated pseudo-random sequence.

[0036] <6> LLaMA-7B is an autoregressive language model developed by Meta AI with 7 billion parameters, primarily used for natural language processing and machine learning research.

[0037] <7> FPGA (Field-Programmable Gate Array): is an integrated circuit whose internal structure can be changed through programming. Unlike traditional ASIC (Application-Specific Integrated Circuit), FPGA allows users to program and debug in the field, and has extremely high flexibility and reconfigurability.

[0038] To facilitate understanding of the embodiments of this application, in conjunction with Figure 1 and Figure 2 Detailed explanation. Figure 1 A block diagram of a large model acceleration system based on random computation is shown in an embodiment of the present invention. Figure 2 This paper illustrates a schematic diagram of the working principle of a large-model acceleration system based on random computation in an embodiment of the present invention. The large-model acceleration system 100 based on random computation in this embodiment includes: a random bitstream encoding module 101, a random computation acceleration module 102, and a random bitstream decoding module 103. The random bitstream encoding module 101 is connected to the random computation acceleration module 102, and the random computation acceleration module 102 is connected to the random bitstream decoding module 103. Through the design of the random bitstream encoding module 101, the random computation acceleration module 102, and the random bitstream decoding module 103, the characteristic of random computation—"implementing complex operations with simple logic gates"—is deeply integrated with the high-computing power requirements of large-model computation, thereby achieving highly parallel, low-power complex operations.

[0039] The random bitstream encoding module 101 is used to encode the weight parameters and input features of a large model from floating-point numbers into a random bitstream of a preset number of bits.

[0040] The Stochastic Bit Stream (SBS) module converts the weight parameters and input features of large models from traditional binary format (floating-point numbers) to a stochastic bit stream, providing a data foundation for subsequent random computation.

[0041] In some embodiments of this application, the random bitstream encoding module includes: a normalization unit, used to normalize the weight parameters and input features of a large model to a normalization interval and save the normalization coefficients; a pseudo-random number generation unit, used to generate uniform random numbers based on a configured feedback function; and an encoding unit, used to, if the normalization interval is 0 to 1, compare the uniform random numbers generated by the pseudo-random number generation unit and output a random bitstream of a preset number of bits; and if the normalization interval is -1 to 1, encode the uniform random numbers generated by the pseudo-random number generation unit based on a bipolar encoding mechanism to obtain the final random bitstream.

[0042] For example, the weight parameters and input features of the large model are normalized using a Python NumPy script, thereby normalizing them to a normalization interval of [0, 1] or [-1, 1]. The normalization coefficients at this time are recorded for subsequent decoding to restore the original numerical range.

[0043] For example, a 16th-order linear feedback shift register (LFSR) can be used as a pseudo-random number generator to achieve high-quality pseudo-randomness with low hardware cost and guaranteed randomness. Its polynomial selection... , as the configured feedback function, where This represents an undetermined variable. By introducing uniformly random numbers, we ensure that the uniformity error of the final generated random bitstream is ≤0.1%, thus avoiding statistical bias from affecting the calculation accuracy.

[0044] In some embodiments of this application, the process of encoding the uniform random number generated by the pseudo-random number generation unit based on the bipolar coding mechanism to obtain the final random bit stream includes: encoding based on the input positive value and the uniform random number to output a random bit stream of a preset number of bits; calculating the positive part probability and the negative part probability based on the input negative number, and comparing the positive part probability, the negative part probability and the uniform random number to generate a positive random bit stream and a negative random bit stream of a preset number of bits respectively; subtracting the negative random bit stream from the generated positive random bit stream to obtain the final random bit stream.

[0045] Specifically, if the input value is within the normalized interval [0, 1], that is, the input value is positive, the input value corresponds to the probability of "1" appearing in the random bit stream. For example, if the input value is 0.6, then the probability of "1" in the preset number of bits, such as 256 bits, is 0.6, and the corresponding 256 bits of random bit stream is 11011001101101100110100110...

[0046] If the entered value is negative, for example, -0.3, first calculate the probability of a positive number. The probability of the negative part is Generate a positive random bit stream using positive probability. Use the negative part probability to generate a negative random bit stream. Then The probability of getting "1" minus The probability of "1" is used to obtain the final random bit stream; that is, for negative numbers, bipolar encoding is used, employing two complementary bits. , The final random bit stream is calculated.

[0047] The random computation acceleration module 102 is used to perform multiplication and addition operations and activation function operations based on the weight parameters and input features converted into a random bit stream to obtain the computation result.

[0048] A dedicated random computation circuit was designed to accelerate the core computations of large models—matrix multiplication and activation function computation—by using a random computation acceleration module, thereby reducing hardware complexity.

[0049] In some embodiments of this application, the random computation acceleration module includes: a multiplication unit, used to input weight parameters and input features encoded as random bit streams into two input AND gates for multiplication to obtain the multiplication result of the random bit stream; an addition unit, used to input the multiplication result of the random bit stream output by the multiplication unit into an OR gate array for operation and output the accumulated result; a ReLU function calculation unit, used to compare the input original random bit stream with the all-zero random bit stream based on a comparator, and output the all-zero random bit stream if the input value is less than or equal to zero; and output the original random bit stream if the input value is greater than zero; and a GELU function calculation unit, used to input the input original random bit stream into an AND gate and an OR gate group to calculate the GELU function.

[0050] Matrix multiplication, a core computational component of large models, traditionally employs numerous multipliers and adders, resulting in excessively high hardware costs. The random computation acceleration module utilizes AND gates for multiplication and OR gates for addition. Specifically, the multiplication unit encodes weight parameters as a random bitstream. and input features The input is fed into a two-input AND gate for operation, i.e. The probability of a "1" in the output random bit stream is This yields the multiplication result of the random bit stream. Weight parameters The probability of getting a "1"; input features The probability of getting a "1". For example, the random bitstream of the 1024 multiplication outputs of a row of the matrix is ​​input into an AND gate array, and the accumulated result is output. From a hardware perspective, the area of ​​a single multiplication unit (AND gate) is only 1 / 8 of that of a traditional INT8 multiplier, and the hardware area of ​​the 1024-way matrix multiplication array is reduced by 60% compared to the traditional architecture, thus lowering hardware costs.

[0051] Meanwhile, lightweight circuits are designed for commonly used activation functions in large models. Specifically, for the ReLU function ( The GELU function compares the input random bitstream with a random bitstream of all zero probabilities using a comparator. If the input probability is less than or equal to zero, it outputs a random bitstream of all zero probabilities; otherwise, it outputs the original random bitstream. Using a piecewise approximation, when hour, Implemented using two AND gates , Addition can be implemented using a single OR gate, that is, by using a combination of AND and OR gates, simplifying the hardware complexity to 1 / 5 of the traditional floating-point GELU unit, thus reducing hardware costs.

[0052] The random bit stream decoding module 103 is used to decode the operation result of the random bit stream into a floating-point number format.

[0053] The random bitstream decoding module converts the output random bitstream calculated by the random computation acceleration module back to the traditional binary format (floating-point format) for use by subsequent layers or the final output.

[0054] In some embodiments of this application, the random bitstream decoding module includes: a statistics unit, used to count the number of ones in the output random bitstream of a preset number of bits based on a counter; a probability calculation unit, used to calculate a probability value based on the count and the preset number of bits; and an inverse normalization unit, used to multiply the probability value output by the probability calculation unit by a normalization coefficient to obtain the final value in floating-point format.

[0055] For example, a 32-bit counter is used to count the number N of "1"s in the output random bitstream. A "right shift" operation is then performed to obtain the probability value. For instance, a 256-bit random bitstream shifted 8 bits to the right is equivalent to N / 256, meaning the output probability = N / 256. The output probability value is then multiplied by the normalization coefficient used during encoding for inverse normalization, thus restoring the original numerical range of the large model and obtaining the final calculation result in floating-point format. The decoding error at a 256-bit SBS length is controlled to ≤0.3%, meeting the accuracy requirements of the large model. Furthermore, if the scenario allows for higher latency (such as offline inference), the length can be increased to 512 bits, reducing the error to below 0.1%, thereby ensuring the accuracy of large model inference.

[0056] In some embodiments of this application, the random bitstream decoding module further includes a calibration unit for calibrating the accumulated result output by the OR gate array by looking up a calibration table based on the calculated probability value, so as to obtain the true accumulated result.

[0057] For example, the number of "1"s N is counted by a 16-bit counter, and the output probability is approximately N / total length. Then, the accumulation result of the OR gate array is calibrated by a calibration unit. The true sum is obtained by looking up a preset calibration table based on the calculated probability value. Finally, an inverse normalization operation is performed based on the true sum to obtain the final value.

[0058] Furthermore, to address the issues of accuracy loss and generality in large models using stochastic computation, this application provides a large model acceleration system based on stochastic computation, which also includes a hardware-software co-optimization module. This module performs three rounds of fine-tuning (learning rate 1e-5, optimizer AdamW) on a target dataset (such as C4 or GLUE) based on the pre-trained large model, adjusting the weight distribution to reduce random bitstream encoding errors, thus achieving model adaptation fine-tuning. A hybrid computation architecture is also designed, retaining traditional binary computation for accuracy-sensitive parts (such as the attention mechanism softmax function), while using a stochastic computation acceleration module for computationally intensive and accuracy-tolerant parts (such as matrix multiplication in the FFN layer of the feedforward network), balancing accuracy and cost. Simultaneously, each module is encapsulated as an ONNX Runtime / TensorRT plugin, allowing the stochastic computation acceleration layer to be specified via configuration files, eliminating the need to modify the core code of the large model and lowering the deployment threshold.

[0059] The large model acceleration system based on random computation provided in this application simplifies the hardware architecture through random computation and optimizes the accuracy through software and hardware co-optimization, resulting in a significant reduction in hardware costs, a substantial improvement in energy efficiency, controllable accuracy loss, and high versatility and deployment flexibility.

[0060] Furthermore, taking "LLaMA-7B large model inference acceleration (text generation scenario)" as an example, a specific example of the large model acceleration system based on random computation provided in this application is illustrated:

[0061] First, the weights of the large model are preprocessed and encoded. Specifically, the LLaMA-7B pre-trained model (FP16 weights) is obtained from a public repository, and the weights are normalized to the [-1, 1] interval using a Python script. The normalized weights are then bipolar encoded, where the positive part generates a 256-bit SBS and the negative part generates a complementary SBS. The final encoded result is stored as a binary file, and the input text is converted into a 4096-dimensional feature vector through a tokenizer, which is also encoded as a 256-bit SBS.

[0062] Secondly, the design of the random computation acceleration module and decoding module on the FPGA side is as follows: For the matrix multiplication unit design of random computation, targeting the FFN layer of the LLaMA-7B feedforward network (4096-dimensional input, 1024-dimensional output), 4096 AND gate multiplication units (parallel processing), 1024 OR gate addition units + a 16-bit counter are designed, with a clock frequency of 100MHz and a computation delay of 2.56μs; For the activation function unit design of random computation, a GELU segmented approximation circuit is implemented, outputting a 256-bit SBS through a combination of random computation comparators, AND gates, and OR gates; For the random bitstream decoding module design, a 32-bit counter is integrated to count the number of "1"s, which is then right-shifted by 8 bits and multiplied by a normalization coefficient (LLaMA-7B weight maximum absolute value 2.5) to restore the FP16 format output.

[0063] Finally, model fine-tuning and accuracy verification were performed. Specifically, LLaMA-7B was fine-tuned using the C4 dataset (1 million texts), the attention layer was frozen, and only the feedforward network FFN layer was fine-tuned. After 3 rounds, the GLUE text classification accuracy improved from 91.8% (without SC adaptation) to 92.2%. In the text generation test, the input was “…”, and the output “…” was randomly calculated to accelerate the process. The semantic consistency with the GPU output reached 98.5%, with an accuracy loss of 0.3%.

[0064] Figure 3 This is a flowchart illustrating the large model acceleration method based on random computation provided in an embodiment of this application. Figure 3 As shown, large model acceleration methods based on stochastic computation include:

[0065] Step S11: Encode the weight parameters and input features of the large model from floating-point numbers into a random bit stream of a preset number of bits;

[0066] Step S12: Perform multiplication and addition operations and activation function operations based on the weight parameters converted into a random bit stream and the input features to obtain the operation result;

[0067] Step S13: Decode the result of the random bit stream operation into floating-point format.

[0068] It should be understood that the large model acceleration method based on random computation provided in this application embodiment can achieve the functions of the large model acceleration system based on random computation described above. For the sake of brevity, it will not be described again here.

[0069] It should also be understood that the module division in the embodiments of this application is illustrative and only represents a logical functional division; in actual implementation, there may be other division methods. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0070] Figure 4 This is a schematic block diagram of the electronic terminal provided in an embodiment of this application. Figure 4 As shown, the electronic terminal 400 includes at least one processor 401, a memory 402, at least one network interface 403, and a user interface 405. The various components in the electronic terminal 400 are coupled together via a bus system 404. It is understood that the bus system 404 is used to implement communication between these components. In addition to a data bus, the bus system 404 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 4 The general will label all buses as bus systems.

[0071] The user interface 405 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0072] It is understood that memory 402 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0073] In this embodiment of the invention, the memory 402 is used to store various types of data to support the operation of the electronic terminal 400. Examples of this data include: any executable program for operation on the electronic terminal 400, such as the operating system 4021 and application programs 4022; the operating system 4021 contains various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 4022 may contain various applications, such as a media player, browser, etc., for implementing various application services. The methods provided in this embodiment of the invention can be included in the application program 4022.

[0074] The methods disclosed in the above embodiments of the present invention can be applied to processor 401, or implemented by processor 401. Processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 401 or by instructions in the form of software. The processor 401 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 401 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 401 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in a memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0075] In an exemplary embodiment, the electronic terminal 400 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to execute the aforementioned method.

[0076] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute... Figure 3 The method in the illustrated embodiment.

[0077] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium storing program code, which, when executed on a computer, causes the computer to perform... Figure 3 The method in the illustrated embodiment.

[0078] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0079] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0080] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0081] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0082] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0083] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0084] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs, DVDs), or semiconductor media (e.g., solid-state disks, SSDs, etc.).

[0085] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] In summary, addressing the technical problems of high hardware cost, low energy efficiency, significant accuracy loss, and poor versatility in existing technologies, this application provides a large-model acceleration system, method, medium, terminal, and program product based on random computation. The system uses a random bitstream encoding module to convert the weight parameters and input features of the large model from floating-point format to a random bitstream of a preset bit length. This converted random bitstream weight parameters and input features are then input to a random computation acceleration module for multiplication, addition, and activation function operations to obtain the computation result. Finally, the calculated result is input to a random bitstream decoding module for decoding, yielding the final computation result in floating-point format. This application reduces hardware costs and simplifies the computational architecture while ensuring the accuracy of large-model inference and training, improving energy efficiency and thus reducing long-term electricity expenses. It also supports mainstream large models (GPT-2, LLaMA-7B, BERT, etc.) and can be switched via software configuration, balancing versatility and deployment flexibility. Furthermore, it lowers the technical threshold, allowing for rapid FPGA-based verification without requiring significant ASIC R&D investment, enabling low-cost deployment for small and medium-sized enterprises. Therefore, this application effectively overcomes the various shortcomings of the prior art and has high industrial application value.

[0087] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A large model acceleration system based on stochastic computation, characterized in that, include: The random bitstream encoding module is used to encode the weight parameters and input features of a large model from floating-point numbers into a random bitstream of a preset number of bits. The random computation acceleration module is used to perform multiplication and addition operations and activation function operations based on the weight parameters and input features converted into a random bit stream to obtain the computation result; The random bitstream decoding module is used to decode the results of random bitstream operations into floating-point number format.

2. The large model acceleration system based on stochastic computation according to claim 1, characterized in that, The random bitstream encoding module includes: The normalization unit is used to normalize the weight parameters and input features of a large model to normalize them to the normalization interval and save the normalization coefficients. A pseudo-random number generation unit is used to generate uniform random numbers based on a configured feedback function; The encoding unit is used to output a random bit stream of a preset number of bits by comparing the uniform random number generated by the pseudo-random number generation unit with the normalization interval of 0 to 1; and to encode the final random bit stream based on the uniform random number generated by the pseudo-random number generation unit according to the bipolar encoding mechanism if the normalization interval is -1 to 1.

3. The large model acceleration system based on stochastic computation according to claim 2, characterized in that, The process of encoding the uniformly random numbers generated by the pseudo-random number generation unit based on the bipolar coding mechanism to obtain the final random bit stream includes: Encode the input positive value and uniform random number to output a random bit stream of a preset number of bits; The positive and negative probabilities are calculated based on the input negative number, and compared with the positive and negative probabilities and uniform random numbers to generate positive and negative random bit streams of a preset number of bits respectively. Subtract the negative random bit stream from the generated positive random bit stream to obtain the final random bit stream.

4. The large model acceleration system based on stochastic computation according to claim 2, characterized in that, The random computation acceleration module includes: The multiplication unit is used to input the weight parameters and input features encoded as a random bit stream into two input AND gates to perform multiplication operations to obtain the multiplication result of the random bit stream; The addition unit is used to input the multiplication result of the random bit stream output by the multiplication unit into the OR gate array for operation and output the accumulated result; The ReLU function calculation unit is used to compare the input original random bit stream with the all-zero random bit stream based on the comparator. If the input value is less than or equal to zero, the all-zero random bit stream is output; if the input value is greater than zero, the original random bit stream is output. The GELU function calculation unit is used to input the original random bit stream into AND gates and OR gates to calculate the GELU function.

5. The large model acceleration system based on stochastic computation according to claim 4, characterized in that, The random bitstream decoding module includes: The statistics unit is used to count the number of ones in the output random bit stream of a preset bit length based on the operation result of the counter. The probability calculation unit is used to calculate the probability value based on the number of counts and the preset number of digits. The inverse normalization unit is used to multiply the probability value output by the probability calculation unit by a normalization coefficient to obtain the final value in floating-point format.

6. The large model acceleration system based on stochastic computation according to claim 5, characterized in that, The random bitstream decoding module also includes a calibration unit, which is used to calibrate the accumulated result output by the OR gate array by looking up the calibration table based on the calculated probability value, so as to obtain the true accumulated result.

7. A method for accelerating large models based on stochastic computation, characterized in that, include: The weight parameters and input features of the large model are encoded from floating-point numbers into a random bit stream of a preset number of bits; Based on the weight parameters converted into a random bit stream and the input features, multiplication and addition operations and activation function operations are performed to obtain the operation result; The result of the operation on the random bit stream is decoded into a floating-point number format.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in claim 7.

9. A computer program product, characterized in that, The computer program product includes computer program code that, when run on a computer, causes the computer to implement the method as described in claim 7.

10. An electronic terminal, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method as described in claim 7.

Images