BGRU quantification method, storage medium and intelligent device
Through fixed-point quantization of the BGRU model, the problems of large computing volume and poor real-time performance are solved, and efficient operation on resource-constrained devices is achieved, power consumption is reduced and real-time performance is improved.
Patent Information
- Application Number
- CN202411318657.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2025-01-10
AI Technical Summary
The BGRU model has large computing volume, high power consumption, and poor real-time performance, making it difficult to apply to edge devices or wearable devices with limited resources.
By performing fixed-point quantization of the input data, output data and weight data of the BGRU model, the storage space and computing resources are reduced, fixed-point calculation is adopted to improve computing efficiency, and when necessary, the fixed-point data is restored to the data format required by the activation function.
Reduces storage space and computing resources, improves computing efficiency, reduces power consumption, supports operation on resource-constrained hardware, and improves real-time performance.
Smart Images
Figure CN119294481A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of neural network technology, and in particular to a BGRU quantization method, storage medium, and intelligent device. Background Art
[0002] With the development of technology, a large number of neural network models have emerged in the fields of text generation, machine translation, speech recognition, image description generation, and video tagging. The BGRU (Bidirectional Gated Recurrent Unit) model, also known as the BGRU neural network model, can capture long-term temporal dependencies and avoid information loss in long sequences. It has been widely used in the above fields. However, the BGRU model itself has a large amount of calculation, resulting in high power consumption and long time consumption, and poor real-time performance, so it is difficult to apply to resource-constrained edge devices or wearable devices.
[0003] The existing technology generally solves this problem by modifying the structure of the neural network model, mainly using the GSU (Gated Simple Unit, gated simplified unit) model or the Transformer model to replace the BGRU module to perform calculations. Although the GSU model can reduce the amount of calculation, its expression ability is limited compared to the BGRU, and it may not be able to capture sufficiently complex dependencies in some complex tasks. At the same time, the GSU model is a lightweight gated structure with relatively weak generalization ability. When faced with data with high diversity and complexity, the operation performance is not ideal. The self-attention mechanism of the Transformer model needs to calculate the global attention weight when processing sequences, which leads to higher computational complexity when the sequence is long. The BGRU model processes the sequence step by step and can better process long sequences. At the same time, the Transformer model does not explicitly retain the order information of the sequence like the BGRU model, so it does not perform well in some tasks that depend on the sequence order. More importantly, the number of parameters of the Transformer model is usually large, and its use is relatively limited in resource-constrained environments. Summary of the invention
[0004] In view of this, the present application provides a BGRU quantization method, storage medium, and intelligent device, which can improve the problems of existing neural network models, such as large storage space and computing resources required, poor real-time performance, and the resulting difficulty in adapting to resource-constrained hardware such as wearable devices.
[0005] A BGRU quantization method provided in this application includes:
[0006] Get the original floating-point value range corresponding to the input data, output data and weight data of each layer of the BGRU model;
[0007] Mapping the data in the original floating-point value domain corresponding to the input data, output data and weight data to the fixed-point value domain to complete the fixed-point quantization of the original floating-point value domain;
[0008] Calculate and obtain the quantization factors corresponding to the input data, output data and weight data;
[0009] Based on the quantization factor, fixed-point calculation is performed to obtain the reset gate, update gate, candidate hidden state and hidden state of each layer of the BGRU model.
[0010] Optionally, the quantization factors corresponding to the input data, output data and weight data are calculated by the following relationship:
[0011] Δ=max(abs(r max ), abs(r min )) / k
[0012] Δ represents the quantization factor, r max Represents the maximum value of the original floating-point value range, r min represents the minimum value of the original floating-point value range, and k represents a preset scaling factor.
[0013] Optionally, completing fixed-point quantization of the original floating-point value range includes:
[0014] Dividing the data of the original floating-point value range by the quantization factor to obtain a quotient, and taking an integer for the quotient;
[0015] The integer quotient is truncated according to the signed type and the unsigned type to limit all data in the original floating-point value domain to the fixed-point value domain.
[0016] Optionally, the data of the original floating-point value domain is quantized into an 8-bit fixed-point value domain, the fixed-point value domain is (-128, 127) and its zero point constraint is 0.
[0017] Optionally, when a fixed-point calculation is performed to obtain any one of the reset gate, the update gate, the candidate hidden state and the hidden state, the BGRU model does not set a corresponding bias parameter.
[0018] Optionally, performing floating point calculations to obtain reset gates, update gates, candidate hidden states, and hidden states of each layer of the BGRU model includes:
[0019] r t =σ(w xr x t +w hr (h t-1 +h t+1 ))
[0020] z t =σ(w xz x t +w hz (h t-1 +h t+1 ))
[0021]
[0022]
[0023] r t 、z t , h t are the reset gate, update gate, candidate hidden state and hidden state at the current time t obtained by floating point calculation, σ is the activation function, w xr and w xz are the weight data corresponding to the input data when calculating the reset gate and the update gate, w hr and w hz is the weight data corresponding to the hidden unit, h t-1 is the hidden state at the previous moment t-1, h t+1 is the hidden state at the next moment t+1, w xh and w hh For calculation The weight data corresponding to the input data.
[0024] Optionally, based on the quantization factor, fixed-point calculation is performed to obtain the reset gate, update gate, candidate hidden state and hidden state of each layer of the BGRU model, including:
[0025] Based on the quantization factors, effective quantization factors corresponding to the reset gate, the update gate, the candidate hidden state and the hidden state are calculated respectively;
[0026] Decomposing the effective quantization factor into corresponding scale factors and offsets; and,
[0027] Based on the scale factor and the offset, fixed-point calculations are performed on the reset gate, update gate, candidate hidden state, and hidden state of each layer of the BGRU model.
[0028] Optionally, the corresponding effective quantization factor is calculated by the following relationship:
[0029] Δ en =Δ y *Δ / 2 -m
[0030] Δ en is the effective quantization factor, Δ is the corresponding quantization factor, Δ yis the quantization factor of the corresponding weight data, and m is the first preset parameter used to adjust the data range when the preset ratio specific point number is used.
[0031] Optionally, the effective quantization factor is decomposed into corresponding scale factors and offsets by the following relationship:
[0032] Δ en =(1>>ε)*μ
[0033] μ is the scale factor and ε is the offset.
[0034] Optionally, based on the scale factor and the offset, a reset gate, an update gate, a candidate hidden state, and a hidden state of each layer of the BGRU model are obtained by fixed-point calculation through the following relationship:
[0035] R t =σ((μ exr *(W xr X t ))>>ε exr )+(μ ehr *(W hr (H t-1 +H t+1 ))>>ε ehr ))
[0036] Z t =σ((μ exz (W xz X t )>>ε exz )+(μ ehz *(W hz (H t-1 +H t+1 ))>>ε ehz ))
[0037]
[0038]
[0039] R t , Z t , H t are the reset gate, update gate, candidate hidden state and hidden state at the current time t, σ is the activation function, W xr and W xz For w xr and w xz Quantized weight data, W hr and W hz For w hr and w hz Quantized weight data, Wxh and W hh For w xh and w hh Quantized weight data, H t-1 For h t-1 The quantized hidden state, H t+1 For h t+1 Quantized hidden state, μ exr , μ ehr , μ exz , μ ehz , μ exh , μ ehh is the corresponding scaling factor, ε exr , ε ehr , ε exz , ε ehz , ε exh , ε ehh , ε h is the corresponding offset, p is a preset bit, and s is a second preset parameter used to adjust the data range when the preset bit number is used.
[0040] Optionally, the method further includes:
[0041] The reset gate, update gate, candidate hidden state and hidden state obtained by fixed-point calculation are subjected to floating-point calculation to transform into the data required for the activation function of the BGRU model.
[0042] The present application provides a storage medium storing a program, which, when executed by a processor, implements the steps of the BGRU quantization method as described in any one of the above items.
[0043] The present application provides an intelligent device, including a processor, which can execute the steps of any of the above BGRU quantization methods; and / or, includes the storage medium as described above.
[0044] As described above, the present application performs fixed-point quantization on the original floating-point value domains corresponding to the input data, output data, and weight data of each layer of the BGRU model, and after completing the fixed-point quantization, performs the calculations of each layer of the BGRU model. That is, the present application converts floating-point decimal calculations into fixed-point quantized integer calculations, which can reduce storage space and computing resources, while reducing the model size, improving computing efficiency to improve real-time performance, and saving power consumption, and supporting deployment on resource-constrained hardware including wearable devices. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 It is a flowchart of a BGRU quantization method according to an embodiment of the present application;
[0046] Figure 2It is a structural diagram of a BGRU quantization device according to an embodiment of the present application. DETAILED DESCRIPTION
[0047] In order to solve the above existing technical problems, the present application provides a BGRU quantization method, storage medium, and intelligent device. These protection themes are based on the same concept, and the principles of solving the problems are basically the same or similar. The implementation methods of each protection theme can refer to each other, and the repetitions will not be repeated.
[0048] In each example provided in the present application, the original floating-point value domain corresponding to the input data, output data and weight data of each layer of the BGRU model is fixed-point quantized, and after the fixed-point quantization is completed, the fixed-point operations of the reset gate, update gate, candidate hidden state and hidden state of each layer of the BGRU model are performed. That is, the present application converts floating-point decimal calculations into fixed-point quantized integer calculations, thereby reducing storage space and computing resources, while reducing the model size, improving computing efficiency to improve real-time performance, and saving power consumption, supporting deployment on resource-constrained hardware including wearable devices, and then restoring the data output after fixed-point quantization to the data required by the activation function, so as to be suitable for subsequent calculations.
[0049] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly described below in conjunction with specific embodiments and corresponding drawings. Obviously, the embodiments described below are only part of the embodiments of the present application, not all of the embodiments. In the absence of conflict, the following embodiments and their technical features can be combined with each other and also belong to the technical solutions of the present application.
[0050] Figure 1 FIG. 1 is a flow chart of a BGRU quantization method according to an embodiment of the present application. Figure 1 As shown, the BGRU quantization method may be referred to as a method, which at least includes the following steps S1 to S5.
[0051] S1: Get the original floating-point value domain corresponding to the input data, output data and weight data of each layer of the BGRU model.
[0052] S2: Map the data in the original floating-point value domain corresponding to the input data, output data and weight data to the fixed-point value domain to complete the fixed-point quantization of the original floating-point value domain.
[0053] In one example, the training data is input into the BGRU model one by one, and the training data is the input data of each layer of the BGRU model; the input data is a set of data with a maximum value and a minimum value. Similarly, the output data and corresponding weight data of each layer of the BGRU model are also a set of data, each with a maximum value and a minimum value. Here, the present application can obtain the corresponding floating point range, which is the original floating point value domain. Then, according to the maximum value and the minimum value, the quantization factors corresponding to the input data, output data and weight data can be calculated by the following relationship.
[0054] Δ=max(abs(r max ), abs(r min )) / k
[0055] Δ represents a quantization factor, which is expressed as a decimal between 0 and 1. max Represents the maximum value of the original floating-point value range, r min represents the minimum value of the original floating-point value domain, and k represents a preset scaling factor. For example, when the data in the original floating-point value domain (which can be called a "floating-point variable") is mapped to an 8-bit interval, that is, 8 bits are used to represent one byte of data, k can be 128, and the relationship is expressed as follows:
[0056] Δ=max(abs(r max ), abs(r min )) / 128
[0057] In the step S2, the fixed-point quantization of the original floating-point value range is completed, including:
[0058] First, the data of the original floating-point value domain is divided by the quantization factor Δ using the round function to obtain a quotient, and the quotient is rounded to an integer, and the integer obtained by rounding can be regarded as the quantized data;
[0059]
[0060] x int represents the nearest integer, and x represents the data in the original floating-point value range.
[0061] Then, the integer quotient is truncated according to the signed type and the unsigned type to limit all data in the original floating-point value domain to the fixed-point value domain.
[0062] Signed types use the highest bit (usually the leftmost bit) to indicate the sign, 0 for a positive number and 1 for a negative number; unsigned types use all bits to indicate the value itself, and there is no sign bit. Take the int type as an example. In most common systems, int is usually 4 bytes (32 bits). For a signed int type, the highest bit is the sign bit, and the remaining 31 bits are used to indicate the size of the value. For an unsigned unsigned int type, all 32 bits are used to indicate the value.
[0063] In one example, all data in the original floating-point value range are restricted to the fixed-point value range by a truncation function, namely, a clamp function, represented by the following relational expression:
[0064] x Q =clamp(-N levels / 2,N levels / 2-1,x int )
[0065] x Q Represents the data in the fixed-point value range obtained after mapping, N levels It represents the number of levels after quantization. The clamp function is a function used to limit a value to a specified range. It accepts three parameters: the value to be limited x int , lower limit of the range - N levels / 2 and upper limit N levels / 2-1. The clamp function ensures that the output value will not be less than the lower limit or greater than the upper limit, thereby ensuring that the output value is always within the specified range. For example, when quantizing an 8-bit interval (i.e., int8 quantization), the quantization range is (-128, 127). This is because the range that can be represented by an 8-bit integer is from -128 to 127 (including -128 and 127), but because a center point (i.e., 0) is required, the actual available range is between -128 and 127, with a total of 256 levels. At this time, the data of the original floating-point value domain is quantized into an 8-bit fixed-point value domain, and the fixed-point value domain is (-128, 127), and its zero point constraint is 0.
[0066] The quantized data is truncated to the fixed-point value domain, that is, the fixed-point quantization of the original floating-point value domain is completed. The so-called quantization can be understood as converting the data of the BGRU model (that is, the data in the original floating-point value domain) from floating-point representation to integer representation. The main goal is to reduce the storage and transmission overhead of the GRU model by converting the model parameters from high-precision floating-point numbers to low-precision integers, thereby reducing the size and computational complexity of the model, while reducing the running time and resource consumption while maintaining the performance and accuracy of the model as much as possible. In actual scenarios, the quantization can convert the BGRU model to run on a specific hardware platform, suitable for image, audio, text and other processing.
[0067] S3: Calculate and obtain the quantization factors corresponding to the input data, output data and weight data.
[0068] This application can be achieved by using the above relationship Δ=max(abs(r max ), abs(r min )) / k, and calculate the quantization factors corresponding to the input data, output data and weight data. The original floating-point value ranges of the input data and output data can be obtained according to the statistics of the actual processing process of the BGRU model, and the original floating-point value ranges of other data can be obtained according to the set parameters of the BGRU model.
[0069] S4: Based on the quantization factor, fixed-point calculation is performed to obtain the reset gate, update gate, candidate hidden state and hidden state of each layer of the BGRU model.
[0070] The network structure of the BGRU model mainly includes input variables, gated hidden states, reset gates, update gates, candidate hidden states, and of course activation functions. Among them, the reset gate helps capture short-term dependencies in the sequence; the update gate helps capture long-term dependencies in the sequence.
[0071] The existing method obtains the reset gate, update gate, candidate hidden state and hidden state through floating point calculation. The calculation process is as follows:
[0072] Step 1: For a given time step t, the input variable is (n is the number of samples, d is the input feature dimension), the hidden state at the previous time step is (h is the number of hidden units), the reset gate r can be calculated by the following relationship: t and update gate z t .
[0073] r t =σ(w xr x t +w hr (h t-1 +h t+1)+b r )
[0074] z t =σ(w xz x t +w hz (h t-1 +h t+1 )+b z )
[0075] in, is the weight parameter corresponding to the input variable, is the weight parameter corresponding to the hidden unit, is the bias parameter. Here, the Sigmoid activation function σ maps the input value to between 0 and 1 to control the degree of information transmission.
[0076] Step 2: Calculate the candidate hidden state at time step t using the following relationship:
[0077]
[0078] in, represents the candidate hidden state, and is the weight parameter corresponding to the input variable, is the bias parameter, and the symbol ⊙ is the product operator. Here, the tanh activation function introduces nonlinear activation for the hidden state and generates new candidate states so that the network can effectively remember or forget information.
[0079] In the candidate hidden state In the example, the reset gate r is used t As a coefficient to control which information in the hidden state of the previous time step is used when processing the input variable. Reset the gate r t Enter variable x when it is close to the minimum value t It dominates, indicating that the information contained in the input of the current time step is less relevant to the previous information; reset gate r t When the input variable x is close to the maximum value t It is highly correlated with the previous long-term information and needs to be comprehensively considered to generate the information for the current time step.
[0080] Step 3. Finally, the hidden state at time step t, i.e. the output, is calculated by the following relationship:
[0081]
[0082] Among them, h t is the hidden state. Whenever the gate z is updated tWhen it approaches the maximum value, the GRU model tends to keep only the old state. At this time, the information from the current time step t is basically ignored, effectively skipping the time step in the dependency chain. On the contrary, when the update gate z t When approaching the minimum value, the new hidden state h t will be close to the candidate hidden state
[0083] In an example of the present application, when the floating-point calculation is converted to a fixed-point calculation to obtain any one of the reset gate, the update gate, the candidate hidden state and the hidden state, the BGRU model does not set the corresponding bias parameter.
[0084] In one example, the step S4 may include the following steps S41 to S43:
[0085] S41: Based on the quantization factors, effective quantization factors corresponding to the reset gate, the update gate, the candidate hidden state and the hidden state are calculated and obtained respectively.
[0086] S42: Decompose the effective quantization factor into corresponding scale factors and offsets.
[0087] S43: Based on the scale factor and the offset, fixed-point calculation is performed to obtain the reset gate, update gate, candidate hidden state and hidden state of each layer of the BGRU model.
[0088] For the reset gate r of BGRU t and update gate z t , this application adopts the symmetric quantization method and expresses the result of the corresponding fixed-point calculation as R t , Z t ; floating point weight w associated with the input data xr 、w xz The quantization factor Δ xr , Δ xz , the weight data is symmetrically quantized accordingly, and the weight of the fixed-point calculation becomes W xr , W xz After these weights are multiplied by the input data, they need to be transformed to the data range required by the activation function lookup table. Based on this, the corresponding effective quantization factor can be calculated through the following relationship:
[0089] Δ en =Δ y *Δ / 2 -m
[0090] Δ en is the effective quantization factor, Δ is the corresponding quantization factor, Δ y is the quantization factor of the corresponding weight data, and m is the first preset parameter used to adjust the data range when the preset ratio specific point number is used.
[0091] The effective quantization factor is decomposed into the corresponding scale factor and offset through the following relationship:
[0092] Δ en =(1>>ε)*μ
[0093] μ is the scale factor and ε is the offset.
[0094] For example, when using a 16-bit interval, the calculated m value is 12, and the reset gate r is calculated. t and update gate z t The corresponding effective quantization factor is Δ exr , Δ exz , the above relationship is expressed as Δ eni =Δ xn *Δ x / 2 -12 , Δ eni is Δ exr When n is r, Δ eni is Δ exz When , n can be z. x is the quantization factor corresponding to the input data, Δ xn It is the quantization factor of the weight data corresponding to the input data.
[0095] In calculating the candidate hidden state When , the corresponding effective quantization factor is Δ ehn , Δ exh and Δ ehh , the above relationship is expressed as follows:
[0096] Δ ehn =Δ hn *Δ h / 2 -12
[0097] Δ exh =Δ xh *Δ x / 2 -12
[0098] Δ ehh =Δ hh *Δ h / 2 -12
[0099] Among them, Δ h is the quantization factor corresponding to the hidden state, Δ hn , Δ xh and Δ hh It is the quantization factor of the weight data corresponding to the hidden state.
[0100] Based on the scale factor and the offset, the present application can obtain the reset gate, update gate, candidate hidden state and hidden state of each layer of the BGRU model by fixed-point calculation through the following relationship:
[0101] R t =σ((μ exr *(W xr X t ))>>ε exr )+(μ ehr *(W hr (H t-1 +H t+1 ))>>ε ehr ))
[0102] Z t =σ((μ exz (W xz X t )>>ε exz )+(μ ehz *(W hz (H t-1 +H t+1 ))>>ε ehz ))
[0103]
[0104]
[0105] R t , Z t , H t are the reset gate, update gate, candidate hidden state and hidden state at the current time t, σ is the activation function, W xr and W xz For w xr and w xz Quantized weight data, W hr and W hz For w hr and w hz Quantized weight data, W xh and W hh For w xh and w hh Quantized weight data, H t-1 For h t-1 The quantized hidden state, H t+1 For h t+1 Quantized hidden state, μ exr , μ ehr , μ exz , μ ehz , μ exh, μ ehh is the corresponding scaling factor, ε exr , ε ehr , ε exz , ε ehz , ε exh , ε ehh , ε h is the corresponding offset, p is a preset bit, and s is a second preset parameter used to adjust the data range when the preset bit number is used.
[0106] For example, when using a 16-bit interval, p can be 16, and the calculated s value is 15. At this time, the hidden state H t The calculation relationship can be expressed as follows:
[0107]
[0108] The meanings of the relevant parameters in this application and the prior art can be referenced to each other. By comparison, it can be seen that the bias parameters in the existing relationship are not required for calculation in this application. as well as The complexity of the BGRU model can be reduced without affecting the flexibility and adaptability of the BGRU model.
[0109] In the above relationship, the role of the activation function is to help regulate the values flowing through each layer of the BGRU neural network and limit it to the domain, thereby helping the BGRU model to capture short-term dependencies in the time series. The activation function of the BGRU model is used to add nonlinear characteristics to the BGRU neural network, so that no matter how many layers the BGRU neural network has, the output of any layer is a nonlinear combination of the input variables of that layer, and the input variables obtained by the next layer are also nonlinear combinations, which is equivalent to the BGRU model being a multi-layer nonlinear model that can learn the features of complex data such as images, audio, and text. In the example of the present application, the domain of the activation function can be relatively fixed, for example (-1, 1). In actual scenarios, the activation function of the BGRU model is not limited to the above-mentioned logical function Sigmoid, but can also be a hyperbolic tangent function Tanh or a linear rectifier function ReLU.
[0110] Because the BGRU model combines information in both the forward and reverse directions to more comprehensively capture the context in the sequence data. At each time step, the BGRU model has two parts, one for processing the forward sequence and the other for processing the reverse sequence. The forward part is responsible for capturing the information of the current time step and before, while the reverse part is responsible for capturing the information of the current time step and after. Therefore, the BGRU model has two sets of hidden states: one set is used to capture the information of the forward sequence, and the other set is used to capture the information of the reverse sequence. According to the aforementioned calculation process, the present application can calculate the hidden states of the forward sequence and the reverse sequence output respectively, and the final hidden state is the connection of these two sets of hidden states or a combination of other methods.
[0111] The aforementioned BGRU quantization method can be applied to scenes such as images, audio, and text. The specific type of input data and its maximum and minimum values can be determined according to the adaptability of the usage scenario.
[0112] In one example, the method may further include:
[0113] Step S5: Perform floating-point calculation on the reset gate, update gate, candidate hidden state and hidden state obtained by fixed-point calculation to transform them into the data required by the activation function of the BGRU model.
[0114] The step S5 can be regarded as the BGRU model restoring the fixed-point quantized data to floating-point variables. The present application can use symmetric quantization to achieve the restoration. The so-called inverse quantization (or "dequantization") process of symmetric quantization is to restore the fixed-point quantized data to floating-point variables.
[0115] In an example of the present application, when performing floating point calculation on any one of the reset gate, update gate, candidate hidden state and hidden state, the BGRU model does not set the corresponding bias parameter. Specifically:
[0116] r t =σ(w xr x t +w hr (h t-1 +h t+1 ))
[0117] z t =σ(w xz x t +w hz (h t-1 +h t+1 ))
[0118]
[0119]
[0120] Among them, rt, zt, ht are the reset gate, update gate, candidate hidden state and hidden state at the current time t obtained by floating point calculation, σ is the activation function, w xr and w xz are the weight data corresponding to the input data when calculating the reset gate and the update gate, w hr and w hz is the weight data corresponding to the hidden unit, h t-1 is the hidden state at the previous moment t-1, h t+1 is the hidden state at the next moment t+1, w xh and w hh For calculation The weight data corresponding to the input data.
[0121] In summary, the present application performs fixed-point quantization on the original floating-point value domains corresponding to the input data, output data, and weight data of each layer of the BGRU model, and after completing the fixed-point quantization, performs the calculations of each layer of the BGRU model, that is, the present application converts the floating-point decimal calculation into fixed-point quantized integer calculation, which can achieve at least the following beneficial effects:
[0122] 1. Improve computing efficiency: Fixed-point quantization uses integers to represent data instead of floating-point numbers. Such representation usually requires less storage space and computing resources because integer operations are simpler and faster to execute than floating-point operations. This is especially important for resource-constrained edge devices because it can improve computing efficiency and reduce power consumption.
[0123] 2. Reduce model size: The fixed-point quantized BGRU model takes up less space in storage because the fixed-point number requires fewer bits. This helps reduce the size of the model, thereby reducing the overhead of model loading, transmission, and storage. This is a significant benefit for the limited storage space on edge devices.
[0124] 3. Save power: Fixed-point operations usually require less energy because the hardware implementation of integer operations is simpler than floating-point operations. Energy efficiency is often an important consideration on mobile devices or edge devices, so using quantized fixed-point operations can reduce power consumption and extend the battery life of the device.
[0125] 4. Improve real-time performance: Since integer operations are usually faster, the fixed-point quantized BGRU model is more likely to achieve real-time performance. This has significant advantages for some application scenarios that require fast response, such as real-time image processing or speech recognition.
[0126] 5. Support for more hardware: Fixed-point quantized BGRU models are usually easier to deploy on different types of hardware. Edge devices have a variety of hardware types, and the fixed-point quantized BGRU model has relatively low computing and storage requirements, making it easier to adapt to various devices.
[0127] 6. Reduced communication overhead: Since the parameters of the BGRU model after fixed-point quantization are smaller, the communication overhead is relatively low during the transmission of the BGRU model to the edge device. This is an important advantage for scenarios where the model is transmitted over the network or updated to the edge device.
[0128] The present application also provides a BGRU quantization device, such as Figure 2 As shown, including:
[0129] An acquisition module 21 is used to obtain the original floating point value domains corresponding to the input data, output data and weight data of each layer of the BGRU model;
[0130] A quantization module 22, used to map data in the original floating-point value domain corresponding to the input data, the output data and the weight data to the fixed-point value domain, so as to complete the fixed-point quantization of the original floating-point value domain;
[0131] A first calculation module 23, used to calculate and obtain quantization factors corresponding to input data, output data and weight data;
[0132] The second calculation module 24 is used to perform fixed-point calculation based on the quantization factor to obtain the reset gate, update gate, candidate hidden state and hidden state of each layer of the BGRU model.
[0133] The specific implementation of each of the above modules can be found in the embodiments of the above-mentioned BGRU quantization method, which will not be described one by one here. Therefore, the BGRU quantization device can also have the beneficial effects that can be achieved by any of the above-mentioned BGRU quantization methods.
[0134] An embodiment of the present application also provides a storage medium on which a program is stored. When the program is executed by a processor, the steps of the BGRU quantization method as in any example are implemented.
[0135] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0136] Since the instructions stored in the storage medium can execute the steps in any BGRU quantization method provided in the embodiments of the present application, the beneficial effects that can be achieved by any BGRU quantization method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.
[0137] An embodiment of the present application also provides an intelligent device or chip, including a memory and a processor, wherein a BGRU quantization program is stored in the memory, and when the BGRU quantization program is executed by the processor, the steps of the BGRU quantization method described in any of the above examples are implemented; and / or, the intelligent device or chip is configured with a storage medium as in the above example, and the processor of the intelligent device or chip loads the storage medium to execute the steps of the BGRU quantization method, thereby achieving the beneficial effects that can be achieved by the BGRU quantization method of the corresponding example of the present application.
[0138] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. For ordinary technicians in this field, all equivalent structural changes made using the contents of this specification and drawings are also included in the patent protection scope of the present application.
[0139] This document uses step codes such as S1, S2, etc., the purpose of which is to express the corresponding content more clearly and concisely, and does not constitute a substantial limitation on the order. When technical personnel in this field implement the specific steps, they may execute S2 first and then S1, etc., but these should all be within the scope of protection of this application.
[0140] Although the terms "first, second", etc. are used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. In addition, the singular forms "one", "an", and "the" are intended to include plural forms as well. The terms "or" and "and / or" are interpreted as inclusive, or mean any one or any combination. Only when the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way, an exception to this definition will occur.
Claims
1. A BGRU quantization method, characterized in that: include: Get the original floating-point value range corresponding to the input data, output data and weight data of each layer of the BGRU model; Mapping the data in the original floating-point value domain corresponding to the input data, output data and weight data to the fixed-point value domain to complete the fixed-point quantization of the original floating-point value domain; Calculate and obtain the quantization factors corresponding to the input data, output data and weight data; Based on the quantization factor, fixed-point calculation is performed to obtain the reset gate, update gate, candidate hidden state and hidden state of each layer of the BGRU model.
2. The method according to claim 1, characterized in that The quantization factors corresponding to the input data, output data and weight data are calculated by the following relationship: Δ=max(abs(r max ),abs(r min )) / k Δ represents the quantization factor, r max Represents the maximum value of the original floating-point value range, r min represents the minimum value of the original floating-point value range, and k represents a preset scaling factor.
3. The method according to claim 1 or 2, characterized in that: The fixed-point quantization of the original floating-point value range is completed, including: Dividing the data of the original floating-point value range by the quantization factor to obtain a quotient, and taking an integer for the quotient; The integer quotient is truncated according to the signed type and the unsigned type to limit all data in the original floating-point value domain to the fixed-point value domain.
4. The method according to claim 3, characterized in that The data of the original floating-point value domain is quantized into an 8-bit fixed-point value domain, and the fixed-point value domain is (-128, 127).
5. The method according to claim 1, characterized in that When fixed-point calculation is performed to obtain any one of the reset gate, update gate, candidate hidden state and hidden state of each layer of the BGRU model, the BGRU model does not set the corresponding bias parameter.
6. The method according to claim 5, characterized in that Based on the quantization factor, fixed-point calculation is performed to obtain the reset gate, update gate, candidate hidden state and hidden state of each layer of the BGRU model, including: Based on the quantization factors, effective quantization factors corresponding to the reset gate, the update gate, the candidate hidden state and the hidden state are calculated respectively; Decomposing the effective quantization factor into corresponding scale factors and offsets; and, Based on the scale factor and the offset, fixed-point calculations are performed on the reset gate, update gate, candidate hidden state, and hidden state of each layer of the BGRU model.
7. The method according to claim 6, characterized in that The corresponding effective quantization factor is calculated by the following relationship: D en =D y *D / 2 -m Δ en is the effective quantization factor, Δ is the corresponding quantization factor, Δ y is the quantization factor of the corresponding weight data, and m is the first preset parameter used to adjust the data range when the preset ratio specific point number is used; The effective quantization factor is decomposed into the corresponding scale factor and offset by the following relationship: D en =(1>>e)*m μ is the scale factor, ε is the offset; Based on the scale factor and the offset, the reset gate, update gate, candidate hidden state and hidden state of each layer of the BGRU model are obtained by fixed-point calculation through the following relationship: R t =σ((μ exr *(W xr X t ))>>e exr )+(μ ehr *(W hr (H t-1 +H t+1 ))>>e ehr )) Z t =σ((μ exz (W xz X t )>>e exz )+(μ ehz *(W hz (H t-1 +H t+1 ))>>e ehz )) R t , Z t , H t are the reset gate, update gate, candidate hidden state and hidden state at the current time t, σ is the activation function, W xr and W xz For w xr and w xz Quantized weight data, W hr and W hz For w hr and w hz Fixed-point quantized weight data, W xh and W hh For w xh and w hh Quantized weight data, H t-1 For h t-1 The quantized hidden state, H t+1 For h t+1 Quantized hidden state, μ exr , μ ehr , μ exz , μ ehz , μ exh , μ ehh is the corresponding scaling factor, ε exr , ε ehr , ε exz , ε ehz , ε exh , ε ehh , ε h is the corresponding offset, p is a preset bit, and s is a second preset parameter used to adjust the data range when the preset bit number is used.
8. The method according to claim 1 or 5, characterized in that: The method further comprises: Perform floating point calculation on the reset gate, update gate, candidate hidden state and hidden state obtained by fixed point calculation to transform them into data required by the activation function of the BGRU model; Perform floating point calculations to obtain the reset gate, update gate, candidate hidden state, and hidden state of each layer of the BGRU model, including: r t =σ(w xr x t +w hr (h t-1 +h t+1 )) z t =σ(w xz x t +w hz (h t-1 +h t+1 )) r t 、z t , h t are the reset gate, update gate, candidate hidden state and hidden state at the current time t obtained by floating point calculation, σ is the activation function, w xr and w xz are the weight data corresponding to the input data when calculating the reset gate and the update gate, w hr and w hz is the weight data corresponding to the hidden unit, h t-1 is the hidden state at the previous moment t-1, h t+1 is the hidden state at the next moment t+1, w xh and w hh For calculation The weight data corresponding to the input data.
9. A storage medium, characterized in that: A program is stored, and when the program is executed by a processor, the steps of the BGRU quantization method according to any one of claims 1 to 8 are implemented.
10. A smart device, characterized in that: comprising a processor, the processor being capable of executing the steps of the BGRU quantization method according to any one of claims 1 to 8; And / or, comprising the storage medium as claimed in claim 9.