FPGA-based high-precision key operator implementation and combination method for BERT network inference
By optimizing the operator module of the BERT model on the FPGA platform and implementing the GELU and Softmax functions using lookup tables and piecewise functions, the problem of deploying the BERT model in edge devices is solved, achieving high-precision and high-performance hardware deployment, adapting to new network structures, and supporting low power consumption and easy deployment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-22
- Publication Date
- 2026-03-27
AI Technical Summary
The deployment of the BERT model on edge devices is limited by its intensive computation and large number of parameters, making it difficult to deploy on devices with limited memory.
Using an FPGA platform, the GELU and Softmax functions are implemented through lookup tables or piecewise functions. Combined with relay registers and parameterized buffer modules, the operator modules of the BERT model are optimized to achieve high-precision and high-performance hardware deployment.
The BERT model was implemented with high precision on the FPGA platform, adapting to new network structures, achieving high parallelism, low power consumption, easy deployment, and supporting edge device applications with limited memory.
Smart Images
Figure CN116484924B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a BERT network inference high-precision key operator implementation based on FPGA and a combination method thereof. BACKGROUND
[0002] Transformer is a deep learning architecture that relies entirely on attention mechanisms. In recent years, models based on Transformer have made outstanding progress in the field of natural language processing. Among them, the deep bidirectional language representation model BERT (Bidirectional Encoder Representation from Transformers) has become the preferred application solution for various language tasks due to its performance advantages.
[0003] The BERT model can be widely used as a language pre-training model due to its excellent performance. However, the intensive computation and energy consumption introduced by these pre-trained language representations greatly hinder the popularization of BERT models on edge devices. Moreover, the large number of parameters in the BERT model also makes it difficult to deploy on mobile platforms and embedded devices with limited memory capacity, further limiting the scope of use of BERT models. SUMMARY
[0004] To solve the above-mentioned technical problems in the prior art, the present application provides a BERT network inference high-precision key operator implementation based on FPGA and a combination method thereof.
[0005] The technical solution of the present application is as follows:
[0006] A BERT network inference high-precision key operator implementation based on FPGA and a combination method thereof are provided, which are used to implement an operator module in a BERT model deployed in FPGA. The operator module includes a GELU function, and the method comprises the following steps:
[0007] The value of the GELU function is pre-stored in the FPGA in the form of a lookup table, and the FPGA queries the lookup table according to the input data to obtain the corresponding GELU function value. Alternatively, the GELU function is sampled to determine a piecewise function that can fit the GELU function, and the horizontal and vertical coordinates of the piecewise points of the piecewise function and the piecewise slope are stored in the FPGA in the form of an array. The FPGA calculates the GELU function according to the function interval of the input data to obtain the corresponding GELU function value by calling the horizontal and vertical coordinates of the piecewise points of the corresponding piecewise function and the piecewise slope.
[0008] In some possible implementation manners, the value of the GELU function is pre-stored in the FPGA in the form of a lookup table, and the method further comprises:
[0009] The GELU function is sampled and calculated by using an external program to obtain the numerical value of the GELU function under different independent variables, a data file is created, and the obtained numerical value of the GELU function is stored in the data file;
[0010] A BRAM is established in the FPGA, and the data file is imported into the BRAM.
[0011] In some possible implementation manners, the GELU function is sampled, and a piecewise function capable of fitting the GELU function is determined, and the piecewise point coordinates and the piecewise slope of the piecewise function are stored in the FPGA in the form of an array, and the method further comprises:
[0012] The calculation format of the external function is determined according to the fixed-point number format requirement of the FPGA implementation;
[0013] The GELU function is sampled by using the external function, and a combination of piecewise linear functions capable of fitting the GELU function is determined;
[0014] The piecewise point coordinates and the piecewise slope of each piecewise function are calculated, the piecewise point coordinates of each piecewise function are stored in the FPGA in the form of an array, and the piecewise slope of each piecewise function is stored in the FPGA in the form of an array.
[0015] In some possible implementation manners, the operator module further comprises a Softmax module, and the method further comprises:
[0016] The calculation format of the external function is determined according to the fixed-point number format requirement of the FPGA implementation, the Softmax function is sampled by using the external function, a combination of piecewise linear functions capable of fitting the Softmax function is determined, the piecewise point coordinates and the piecewise slope of each piecewise function are calculated, the piecewise point coordinates of each piecewise function are stored in the FPGA in the form of an array, and the piecewise slope of each piecewise function is stored in the FPGA in the form of an array;
[0017] The parameterized buffer module, the vector comparison module, the first vector subtraction module, the exponential function module, the first vector summation module and the first vector division module are sequentially connected in the FPGA, the parameterized buffer module is configured to dynamically adjust the buffer size according to the length of the input vector, the vector comparison module is configured to select the maximum value of the whole set of input vectors through a binary comparison tree and output, the first vector subtraction module is configured to subtract the maximum value from the whole set of input vectors in turn according to the maximum value obtained by the vector comparison module and output, the exponential function module is configured to take the input as the power and realize the mapping of the exponential function by using the stored segmented function combination, the first vector summation module is configured to sum the values and output according to the output of the exponential function module, and the first vector division module is configured to take the output of the first vector summation module as the denominator, take each item of the input vector as the numerator, perform division and output.
[0018] Each module is also configured to send an activation signal to the module connected to it after completing its own work and outputting valid data, and the remaining modules except the parameterized buffer module are also configured to start working after receiving the activation signal.
[0019] In some possible implementation manners, the method further includes:
[0020] The relay registers are arranged in the FPGA, the relay registers include three groups, one group of the relay registers is connected with the parameterized buffer module and used for bearing input caching, one group of the relay registers is connected between the first vector subtraction module and the exponential function module and used as intermediate caching, and one group of the relay registers is connected with the output end of the first vector division module and used for pipeline output control shaping of the first vector division module.
[0021] In some possible implementation manners, the operator module further includes a layer normalization module, and the method further includes:
[0022] The mean module, the variance module, the second vector subtraction module, the second vector summation module and the second vector division module are arranged in the FPGA, the inputs of the mean module, the variance module and the second vector subtraction module are connected with each other, the output of the mean module is connected with the variance module and the second vector subtraction module respectively, the output of the variance module is connected with the second vector division module, the output of the second vector subtraction module is connected with the second vector division module, the second vector summation module is connected with the mean module and the variance module respectively, the input vector is synchronously sent to the mean module, the variance module and the second vector subtraction module, the mean module is configured to perform summation and division according to the input vector, and the mean of the input vector is obtained and output, the variance module is configured to perform subtraction, power, root extraction and division in sequence according to the input vector and the input of the mean module, and the variance of the input vector is obtained and output, the second vector summation module is configured to provide instantiation function when the mean module and the variance module are calculated, the second vector subtraction module is configured to subtract the input vector from the mean and output, and the second vector division module is configured to divide the output of the second vector subtraction module by the variance and output.
[0023] Each module is also configured to send an activation signal to the module connected with the output after completing the work and outputting valid data, and is also configured to start working after receiving the activation signal.
[0024] The main advantages of the technical scheme of the present application are as follows:
[0025] The FPGA-based BERT network inference high-precision key operator implementation and combination method can realize high-precision hardware of deploying the BERT model on the FPGA platform, and can consider high precision, high performance and high universality in operator sorting, operator selection and operator implementation, can fully exert the high performance of the BERT model for natural language processing tasks, and can ensure that the BERT model can be deployed and applied in edge devices with limited memory capacity; and the reconfigurability of the FPGA platform can make the model quickly adapt to new network architectures, realize high parallelism, low power consumption and easy deployment of the model acceleration. BRIEF DESCRIPTION OF DRAWINGS
[0026] The drawings described herein are used to provide further understanding of the embodiments of the present application, and form a part of the present application. The schematic embodiments of the present application and the description thereof are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:
[0027] Figure 1 It is a schematic diagram of the core structure of an existing BERT network model;
[0028] Figure 2An implementation principle schematic diagram of a GELU function according to an embodiment of the present application;
[0029] Figure 3 An implementation principle schematic diagram of another GELU function according to an embodiment of the present application;
[0030] Figure 4 An implementation principle schematic diagram of a Softmax module according to an embodiment of the present application; Figure 1 An attention module and a Softmax module in a BERT network model shown in the figure are in a relative relationship;
[0031] Figure 5 An implementation structure schematic diagram of a Softmax module according to an embodiment of the present application;
[0032] Figure 6 An implementation structure schematic diagram of a layer normalization module according to an embodiment of the present application. DETAILED DESCRIPTION
[0033] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below in combination with specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.
[0034] The technical solutions provided by the embodiments of the present application will be described in detail below in combination with the drawings.
[0035] Referring to Figure 1 The core structure of a BERT model (Bidirectional Encoder Representation from Transformers) mainly includes an attention module, a layer normalization module and a feed-forward neural network. The attention module is used to realize the function of data recoding, including matrix operation and a Softmax function; the layer normalization module avoids gradient disappearance and gradient explosion in language tasks through reprocessing of data distribution; and the feed-forward neural network includes matrix operation and a GELU activation function. In actual application, the BERT model is greatly limited in hardware deployment at the edge end due to its super large size and the characteristics of intensive calculation and memory access.
[0036] In an embodiment of the present application, considering that FPGA (Field Programmable Gate Array) has more advantages than CPU (Central Processing Unit) and GPU (graphics processing unit) in parallelism, power consumption, cost-effectiveness, and reconfigurability, if the operators in the BERT model can be accelerated based on FPGA, the high performance of the BERT model for natural language processing tasks can be fully utilized; and the reconfigurability of the FPGA platform can quickly adapt to new network architectures, achieving high parallelism, low power consumption, and easy deployment.
[0037] To achieve the above purpose, an embodiment of the present application provides a BERT network inference high-precision operator implementation and combination method based on FPGA, which is used to implement an operator module in a BERT model deployed in an FPGA, the operator module including a GELU function, and the method includes:
[0038] The values of the GELU function are pre-stored in the FPGA in the form of a lookup table, and the FPGA queries the lookup table according to input data to obtain the corresponding GELU function value; or the GELU function is sampled to determine a piecewise function that can fit the GELU function, and the horizontal and vertical coordinates of the piecewise points of the piecewise function and the piecewise slope are stored in the FPGA in the form of an array, and the FPGA calls the horizontal and vertical coordinates of the piecewise points of the corresponding piecewise function and the piecewise slope to calculate the GELU function according to the function interval of the input data to obtain the corresponding GELU function value.
[0039] Specifically, the values of the GELU function are pre-stored in the FPGA in the form of a lookup table, further including:
[0040] The GELU function is sampled and calculated by an external program to obtain the numerical values of the GELU function under different independent variables, a data file is created, and the obtained numerical values of the GELU function are stored in the data file;
[0041] A BRAM is established in the FPGA, and the data file is imported into the BRAM.
[0042] Reference Figure 2 When the numerical value of the GELU function needs to be calculated, the FPGA can obtain the corresponding GELU function value by looking up the corresponding lookup table according to the input data. Figure 2 In the figure, clk represents a pulse signal, rst represents a reset signal, enable represents an enable signal, and address mapping represents address mapping.
[0043] The GELU function is implemented in the FPGA based on a look-up table, so that the operation resources of the system can be effectively saved, and the parallelism of the look-up table is good.
[0044] Further, in an embodiment of the present application, the GELU function can also be implemented based on a polyline fitting method, and the fitting of the GELU function is specifically implemented by combining segmented linear functions.
[0045] The GELU function is implemented in the FPGA based on a polyline fitting method, which mainly completes sampling of the GELU function by an external function calculation, so as to determine the fitting expression of the function, and then the FPGA completes the non-linear function mapping according to the input based on the coefficients calculated by the external function.
[0046] Specifically, in an embodiment of the present application, the GELU function is sampled to determine a segmented function capable of fitting the GELU function, and the segmented point coordinates and the segmented slope of the segmented function are stored in the FPGA in the form of an array, and further comprising:
[0047] The calculation format of the external function is determined according to the fixed-point number format requirement of the FPGA;
[0048] The GELU function is sampled by the external function to determine a combination of segmented linear functions capable of fitting the GELU function;
[0049] The segmented point coordinates and the segmented slope of each segmented function are calculated, the segmented point coordinates of each segmented function are stored in the FPGA in the form of an array, and the segmented slope of each segmented function is stored in the FPGA in the form of an array.
[0050] Further, referring to Figure 3 In order to implement the calculation of the GELU function in the FPGA, in an embodiment of the present application, a subtraction module (subtract), a multiplication module (multiply) and an addition module (Add) are provided in the FPGA, the subtraction module, the multiplication module and the addition module are sequentially connected, the subtraction module is configured to obtain the distance from the segmented point according to the interval of the input data by subtraction, the multiplication module is configured to multiply the slope of the corresponding stage by the distance obtained by the subtraction module to obtain the growth reflected by the difference, and the addition module is configured to add the growth calculated by the multiplication module to the intercept of the corresponding segmented point and output.
[0051] The FPGA calls the segmented point horizontal and vertical coordinates of the corresponding segmented function stored in the interval point table, the segmented slope stored in the slope table, the subtraction module, the multiplication module and the addition module according to the function interval where the input data is located, and calculates the GELU function to obtain the corresponding GELU function value.
[0052] With reference to Figure 4 In addition to the GELU function, the operator module of the BERT model also includes a Softmax module. The Softmax module adopts probability calculation and is used for processing the parameter matrix after feature extraction of the neural network to determine the probability value of the input content belonging to each class. The Softmax model is a generalization of the Logistic regression model on the multi-classification problem. When the number of classes is two, the Softmax regression degenerates into the Logistic regression, that is, the Softmax regression is a general form of the Logistic regression.
[0053] With reference to Figure 5 In an embodiment of the present application, when the operator module also includes the Softmax module, the FPGA-based BERT network inference high-precision key operator implementation and combination method further includes:
[0054] According to the fixed-point number format requirement of the FPGA implementation, the calculation format of the external function is determined, the Softmax function is sampled by using the external function, the combination of the segmented linear functions that can fit the Softmax function is determined, the segmented point horizontal and vertical coordinates of each segmented function are calculated, the segmented point horizontal and vertical coordinates of each segmented function are stored in the FPGA in the form of an array, and the segmented slope of each segmented function is stored in the FPGA in the form of an array.
[0055] The parameterized buffer module (Buffer), the vector comparison module (Vmax), the first vector subtraction module (Vsub), the exponential function module (Vexp), the first vector summation module (Vsum) and the first vector division module (Vdiv) are sequentially connected in the FPGA. The parameterized buffer module is configured to dynamically adjust the buffer size according to the length of the input vector. The vector comparison module is configured to select the maximum value of the entire input vector group by using a binary comparison tree and output the maximum value. The first vector subtraction module is configured to subtract the maximum value from the entire input vector group in sequence and output the result according to the maximum value obtained by the vector comparison module. The exponential function module is configured to use the input as the power to realize the mapping of the exponential function by using the stored segmented function combination. The first vector summation module is configured to sum and output the values according to the output of the exponential function module. The first vector division module is configured to use the output of the first vector summation module as the denominator and use each item of the input vector as the numerator to perform division and output the result.
[0056] Each module is also configured to send an activation signal to the module connected to its output after completing its own work and outputting valid data, and the remaining modules except the parameterized buffer module are configured to start working after receiving the activation signal.
[0057] As the output function of the attention mechanism score, the Softmax has higher requirements for the precision control and numerical range of the exponential function, therefore, in an embodiment of the present application, a piecewise function is used to fit the Softmax function; at the same time, multiple function modules are configured in the FPGA to realize the exponential operation of the Softmax in a pipeline manner, which can further improve the optimization flexibility of the model.
[0058] In order to ensure higher precision, according to the specific form of the Softmax function, the fitting function is composed of five one-order functions connected.
[0059] In an embodiment of the present application, the parameterized buffer module is used to save the input data, and the length of the input data is input as a separate parameter, and the FPGA compares the received data with the recorded length parameter every time a piece of data is received, if the current storage length reaches the preset length, it indicates that the input of this group of data is complete, and the subsequent function module is enabled to start working.
[0060] By setting the parameterized buffer module, input data whose length exceeds the support of a single cycle can be saved, so that any length of vector can be supported for operation, thereby being applicable to application scenarios such as language data with large length variation.
[0061] In an embodiment of the present application, the sequential calling of each function module is realized through the control signal between modules. Specifically, when a function module completes its work and outputs valid data, it will send a signal to activate the next function module connected to its output, and when the subsequent function module does not receive the activation signal, it will not start working; when the first batch of data flows through each function module in turn and is output as the first group of data, the pipeline working mode is automatically entered.
[0062] Reference Figure 5 Further, in an embodiment of the present application, the BERT network inference high-precision key operator implementation and combination method based on FPGA further comprises:
[0063] The relay register is arranged in the FPGA, and the relay register includes three groups, one group of relay registers is connected with the parameterized buffer module and is used to bear the input cache, one group of relay registers is connected between the first vector subtraction module and the exponential function module and is used as the intermediate cache, and one group of relay registers is connected with the output end of the first vector division module and is used for pipeline output control shaping of the first vector division module.
[0064] In order to support the pipeline design of the whole workflow, the relay register is arranged in the FPGA, and the relay register is configured to store the current result at each intermediate stage. Therefore, after each module operation is completed, the assignment of the relay register also needs to be additionally arranged.
[0065] Reference Figure 1 In addition to the GELU function and the Softmax module, the operator module of the BERT model also includes a layer normalization module.
[0066] Reference Figure 6 When the operator module also includes the layer normalization module, the FPGA-based BERT network inference high-precision key operator implementation and the combination method thereof further include the following in an embodiment of the application:
[0067] The mean module, the variance module, the second vector subtraction module, the second vector sum module and the second vector division module are arranged in the FPGA, the inputs of the mean module, the variance module and the second vector subtraction module are connected with each other, the output of the mean module is connected with the variance module and the second vector subtraction module respectively, the output of the variance module is connected with the second vector division module, the output of the second vector subtraction module is connected with the second vector division module, the second vector sum module is connected with the mean module and the variance module respectively, the input vector is synchronously sent to the mean module, the variance module and the second vector subtraction module, the mean module is configured to perform summation and division according to the input vector, to obtain the mean of the input vector and output, the variance module is configured to perform subtraction, power, root extraction and division according to the input vector and the input of the mean module, to obtain the variance of the input vector and output, the second vector sum module is configured to provide instantiation function when the mean module and the variance module are calculated, the second vector subtraction module is configured to subtract the mean from the input vector and output, and the second vector division module is configured to divide the output of the second vector subtraction module by the variance and output.
[0068] Each module is also configured to send an activation signal to the module connected with the output thereof after completing the work and outputting the valid data, and is also configured to start working after receiving the activation signal.
[0069] In an embodiment of the present application, the FPGA performs normalization on the input vector data based on the above-mentioned function modules, so that the processed data is limited within a certain range, and the adverse data influence caused by individual samples can be reduced, thereby balancing the influence of different dimensions and dimension units on data analysis.
[0070] In an embodiment of the present application, for the layer normalization module, the sequential calling of each function module is realized through the control signals between modules. Specifically, when a function module completes its work and outputs valid data, a signal is sent to activate the next function module connected to the output of the function module. When the subsequent function module does not receive the activation signal, it will not start working. When the first batch of data flows through each function module in turn and is output as the first group of data, the pipeline working mode is automatically entered.
[0071] In order to support the pipeline design of the whole workflow, a relay register is set in the FPGA, which is configured to store the current result at each intermediate stage. Therefore, after each module operation is completed, the assignment of the relay register also needs to be set additionally.
[0072] Specifically, for the layer normalization module, the relay register includes 6 groups, and the 6 groups of relay registers are respectively used for input shaping, average value calculation pipeline, summation module relay, subtraction calculation relay, variance calculation pipeline and division calculation pipeline.
[0073] The BERT network inference high-precision key operator implementation and combination method based on FPGA provided in an embodiment of the present application can realize high-precision hardware of the BERT model deployed on the FPGA platform, and can balance high precision, high performance and high universality in operator sorting, operator selection and operator implementation, can fully exert the high performance of the BERT model for natural language processing tasks, and ensure that the BERT model can be deployed and applied in edge devices with limited memory capacity; and the reconfigurability of the FPGA platform enables the model to quickly adapt to new network architectures, realizes high parallelism, low power consumption and easy deployment of the model acceleration.
[0074] It should be noted that, in this article, relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. In addition, "front", "rear", "left", "right", "upper", "lower", and the like, are all referred to the placement state shown in the drawings.
[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not limited thereto; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. An implementation and combination method of high-precision key operators for BERT network inference based on FPGA, characterized in that, The method is used to implement the deployment of operator modules in a BERT model in an FPGA. The operator modules include GELU functions. The method includes: The values of the GELU function can be pre-stored in the FPGA in the form of a lookup table. The FPGA can then look up the lookup table based on the input data to obtain the corresponding GELU function value. Alternatively, the GELU function can be sampled to determine the piecewise function that can fit the GELU function. The x and y coordinates and the slope of the piecewise function's segment points can be stored in the FPGA as an array. The FPGA can then use the x and y coordinates and the slope of the corresponding piecewise function to calculate the GELU function based on the function interval in which the input data is located to obtain the corresponding GELU function value. The operator module further includes a Softmax module, and the method further includes: The calculation format of the external function is determined according to the fixed-point format requirements of the FPGA implementation. The external function is used to sample the Softmax function to determine the combination of piecewise linear functions that can fit the Softmax function. The x and y coordinates and the slope of each piecewise function are calculated. The x and y coordinates of each piecewise function are stored in the FPGA in the form of an array, and the slope of each piecewise function is stored in the FPGA in the form of an array. In the FPGA, a parameterized buffer module, a vector comparison module, a first vector subtraction module, an exponential function module, a first vector summation module, and a first vector division module are set up in sequence. The parameterized buffer module is configured to dynamically adjust the buffer size according to the length of the input vector. The vector comparison module is configured to select the maximum value of the entire input vector group through a binary comparison tree and output it. The first vector subtraction module is configured to subtract the maximum value of the entire input vector group from the maximum value obtained by the vector comparison module and output it. The exponential function module is configured to use the input as the power and realize the mapping of the exponential function by using the combination of stored piecewise functions. The first vector summation module is configured to sum the values according to the output of the exponential function module and output them. The first vector division module is configured to use the output of the first vector summation module as the denominator and the terms of the input vector as the numerator, perform division and output it. Each module is also configured to send an activation signal to the module to which it is connected after completing its own work and outputting valid data. The other modules, except for the parameterized buffer module, are also configured to start working after receiving the activation signal.
2. The implementation and combination method of high-precision key operators for BERT network inference based on FPGA according to claim 1, characterized in that, The values of the GELU function are pre-stored in the FPGA in the form of a lookup table, further including: An external program is used to sample and calculate the GELU function to obtain the values of the GELU function under different independent variables. A data file is created and the obtained values of the GELU function are stored in the data file. Create a BRAM in the FPGA and import the data file into the BRAM.
3. The implementation and combination method of high-precision key operators for BERT network inference based on FPGA according to claim 1, characterized in that, The GELU function is sampled to determine piecewise functions that can fit the GELU function. The x and y coordinates of the segment points and the slope of each segment are stored in an array in the FPGA. Further steps include: The calculation format of the external function is determined according to the fixed-point format requirements of the FPGA implementation; By sampling the GELU function using an external function, a combination of linear functions that can fit the piecewise segments of the GELU function is determined; Calculate the x and y coordinates of the segment points and the segment slope for each piecewise function. Store the x and y coordinates of the segment points and the segment slope for each piecewise function in the FPGA as an array.
4. The implementation and combination method of high-precision key operators for BERT network inference based on FPGA according to claim 1, characterized in that, The method further includes: In the FPGA, relay registers are set up, including three sets: one set of relay registers is connected to the parameterized buffer module to serve as input buffer; one set of relay registers is connected between the first vector subtraction module and the exponential function module to serve as intermediate buffer; and one set of relay registers is connected to the output of the first vector division module to control and shape the pipelined output of the first vector division module.
5. The implementation and combination method of high-precision key operators for BERT network inference based on FPGA according to claim 1, characterized in that, The operator module further includes a layer normalization module, and the method further includes: In the FPGA, a mean module, a variance module, a second vector subtraction module, a second vector summation module, and a second vector division module are configured. The inputs of the mean module, the variance module, and the second vector subtraction module are interconnected. The output of the mean module is connected to both the variance module and the second vector subtraction module. The output of the variance module is connected to the second vector division module. The output of the second vector subtraction module is also connected to the second vector division module. The second vector summation module is connected to both the mean module and the variance module. The input vector is synchronously sent to the mean module, the variance module, the second vector summation module, and the second vector division module. The variance module and the second vector subtraction module are configured such that the mean module is configured to perform summation and division on the input vector to obtain the mean of the input vector and output it; the variance module is configured to perform subtraction, exponentiation, square root and division on the input vector and the input of the mean module in sequence to obtain the variance of the input vector and output it; the second vector summation module is configured to provide instantiation function when the mean module and the variance module are calculated; the second vector subtraction module is configured to subtract the mean from the input vector and output it; and the second vector division module is configured to divide the output of the second vector subtraction module by the variance and output it. Each module is also configured to send an activation signal to the module it outputs after completing its own work and outputting valid data, and is also configured to start working after receiving the activation signal.