GRU quantification method based on Gaussian distribution, storage medium and intelligent device
Through the GRU quantization method based on Gaussian distribution, the variance of the training data is used to perform domain reconstruction of Gaussian sampling and activation function, which solves the problems of poor adaptability and insufficient calculation accuracy of GRU quantization in the prior art, and realizes the Higher numerical calculation accuracy.
Patent Information
- Application Number
- CN202411208302.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-24
AI Technical Summary
The existing GRU quantization method is difficult to apply to non-fixed input data, and the calculation accuracy is insufficient, especially when the training event changes, the sample variance fluctuates greatly, which cannot effectively improve the numerical calculation accuracy.
By obtaining the variance of the training data, performing Gaussian sampling, reconstructing the domain of the activation function into the Gaussian sample set, and interpolation calculation is performed to achieve quantization, which is suitable for non-fixed input data and improving calculation accuracy.
This method can be applied to non-fixed input data, improves the numerical calculation accuracy, reflects the continuous distribution of samples through Gaussian distribution, and enhances the mapping accuracy of the activation function.
Smart Images
Figure CN119180301A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of neural network technology, and in particular to a GRU quantization method based on Gaussian distribution, a storage medium, and an intelligent device. Background Art
[0002] The GRU (Gated Recurrent Unit) model, also known as the GRU neural network model, is a variant of the traditional RNN (Recurrent Neural Network) model, which aims to solve the gradient vanishing and gradient exploding problems in the traditional RNN model. The key difference between the GRU model and the traditional RNN model is that the traditional RNN model is designed to solve the temporal dependencies between data. Generally, it can learn the relationship between adjacent time step data points (short-term) in the sequence very well, but it becomes unstable for learning long-term dependencies or dependencies in long sequences; while the GRU model supports the gating of hidden states, that is, the GRU model controls the flow of information by introducing update gates and reset gates, so as to better capture the dependencies in long sequences, which means that the GRU model has a special mechanism to determine when the hidden state should be updated and reset. The GRU model can achieve the same effect as the traditional RNN model with a simpler structure and faster calculation speed. Therefore, the GRU model has gradually been widely used. When the GRU model is applied to application scenarios with low power consumption and high real-time requirements, the industry generally reduces the amount of calculation through GRU quantization (that is, converting floating-point data into fixed-point data), thereby enabling the GRU model to achieve higher reasoning efficiency.
[0003] The existing GRU quantization method usually sets the activation function domain with fixed upper and lower limits, constructs the activation function table with uniform distribution domain, and uses the average interpolation method to find the activation function value to calculate the quantization value of each unit of the GRU model. However, the activation function domain with fixed upper and lower limits cannot be applied to non-fixed input data, for example, it cannot be applied to scenarios where the sample variance fluctuates greatly due to changes in training events. The activation function table with uniform distribution domain cannot produce high-precision mapping results for dense independent variable sets near the mean. When the average interpolation method is used to find the activation function value, the distance between the interpolation points is the same, and the different distances between the interpolation points cannot be used to improve the numerical calculation accuracy. In summary, the existing GRU quantization method is difficult to apply to non-fixed input data, and the calculation accuracy needs to be improved. Summary of the invention
[0004] In view of this, the present application provides a GRU quantization method, storage medium, and intelligent device based on Gaussian distribution, which can improve the problem that the existing GRU quantization method is difficult to apply to non-fixed input data and the calculation accuracy needs to be improved.
[0005] The present application provides a GRU quantization method based on Gaussian distribution, including:
[0006] Obtain training data for input into the GRU model, and calculate the variance of the training data;
[0007] Perform Gaussian sampling according to the variance of the training data to obtain a Gaussian sample set;
[0008] Sorting the elements in the Gaussian sample set;
[0009] Reconstructing the domain of the activation function of the GRU model to limit the domain of the activation function of the GRU model to the sorted Gaussian sample set;
[0010] Performing interpolation calculation on the activation function after reconstructing the definition domain to obtain a corresponding value range list;
[0011] Mapping the elements in the range list to a fixed-point range list to complete the quantization of the GRU model;
[0012] Based on the quantized GRU model, the reset gate, update gate, candidate hidden states, and hidden states are calculated.
[0013] Optionally, the variance of the training data is calculated according to the maximum and minimum values of the training data by using equation 1:
[0014] σ=(I max -I min ) 2 Relation 1
[0015] The σ represents the variance of the training data, and the I max represents the maximum value of the training data, the I min Represents the minimum value of the training data.
[0016] Optionally, the Gaussian sample set is obtained through equation 2:
[0017]
[0018] The N(0,σ) represents a Gaussian distribution with a mean of zero and a variance equal to the calculated variance of the training data. t ~N(0,σ),ι=1t2,3,...,T represents non-repetitive sampling T times from the N(0,σ), and the D is a set of Gaussian samples obtained by the T times of sampling.
[0019] Optionally, sorting the elements in the Gaussian sample set includes:
[0020] The elements in the Gaussian sample set are sorted in ascending order.
[0021] Optionally, the elements in the Gaussian sample set are sorted according to equation 3:
[0022] Q=Sort(D)=[q t ], t=1, 2, 3, ..., T Relation 3
[0023] Q represents a sorted Gaussian sample set, Sort(D) represents a sorting function used for D, and q t represents the t-th element in the sorted Gaussian sample set.
[0024] Optionally, limiting the domain of the activation function of the GRU model to the sorted Gaussian sample set includes:
[0025] The domains of the Sigmoid activation function and the Tanh activation function of the GRU model are limited to the sorted Gaussian sample set.
[0026] Optionally, interpolation calculation is performed on the activation function after the domain reconstruction to obtain a corresponding range list, including:
[0027] The range list A of the Sigmoid activation function after domain reconstruction is calculated by equation 4, and the range list B of the Tanh activation function after domain reconstruction is calculated by equation 5:
[0028] A=Sigmoid(Q)=[Sigmoid(q t )], t = 1, 2, 3, ..., T Relation 4
[0029] B=Tanh(Q)=[Tanh(q t )], t = 1, 2, 3, ..., T Relation 5
[0030] Optionally, the elements in the range list are mapped to a fixed-point range list by using equation 6 to complete the quantization of the GRU model:
[0031]
[0032] The i represents the quantization factor, b is the quantization bit width, and A q represents the fixed-point value range list mapped by the value range list A, the B q Represents the fixed-point value range list mapped by the value range list B.
[0033] The present application provides a storage medium storing a program, which, when executed by a processor, implements the steps of the Gaussian distribution-based GRU quantization method as described in any one of the above items.
[0034] The present application provides an intelligent device, comprising a processor, wherein the processor can execute any step of the above-mentioned GRU quantization method based on Gaussian distribution; and / or, comprises the storage medium as described above.
[0035] As described above, the present application performs Gaussian sampling according to the variance of the training data to obtain a Gaussian sample set, and limits the domain of the activation function of the GRU model to the Gaussian sample set, that is, the upper and lower limits of the domain of the activation function are adaptively set according to the variance of the training data, rather than fixed, and therefore can be applicable to non-fixed input data; and, the present application performs sampling based on the Gaussian distribution to achieve domain reconstruction of the activation function. Since the Gaussian distribution more realistically reflects the continuous distribution of samples, it can produce high-precision mapping results for dense independent variable sets close to the mean, for example, thereby improving the accuracy of numerical calculations; further, interpolation calculations are performed on the sample set obtained based on the Gaussian distribution, which is equivalent to calculating the value of the activation function based on a discrete empirical interpolation algorithm. The distance between the interpolation points reflects the true sample distribution, which is beneficial to improving the accuracy of numerical calculations. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 is a flow chart of a GRU quantization method based on Gaussian distribution according to an embodiment of the present application;
[0037] Figure 2 It is a structural diagram of a GRU quantization device according to an embodiment of the present application. DETAILED DESCRIPTION
[0038] In order to solve the above-mentioned existing technical problems, the present application provides a GRU quantization method based on Gaussian distribution, a storage medium, and an intelligent device. These protection themes are based on the same concept, and the principles of solving the problems are basically the same or similar. The implementation methods of each protection theme can refer to each other, and the repeated parts will not be repeated.
[0039] In each example provided in the present application, Gaussian sampling is performed according to the variance of the input training data to obtain a Gaussian sample set, and the domain of the activation function of the GRU model is limited to the Gaussian sample set, that is, the upper and lower limits of the domain of the activation function are adaptively set according to the variance of the training data, rather than fixed, so as to be suitable for non-fixed input data; and, sampling is performed based on the Gaussian distribution to achieve domain reconstruction of the activation function. The Gaussian distribution more realistically reflects the continuous distribution of the samples, thereby producing high-precision mapping results for a dense independent variable set close to the mean, for example, to improve the accuracy of numerical calculations; further, interpolation calculations are performed on the sample set obtained based on the Gaussian distribution, which is equivalent to calculating the value of the activation function based on a discrete empirical interpolation algorithm. The distance between the interpolation points is a reflection of the true sample distribution, which is beneficial to improving the accuracy of numerical calculations.
[0040] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly described below in conjunction with specific embodiments and corresponding drawings. Obviously, the embodiments described below are only part of the embodiments of the present application, not all of the embodiments. In the absence of conflict, the following embodiments and their technical features can be combined with each other and also belong to the technical solutions of the present application.
[0041] Figure 1 is a flow chart of a GRU quantization method based on Gaussian distribution according to an embodiment of the present application. Figure 1 As shown, the GRU quantization method based on Gaussian distribution can be referred to as "GRU quantization method" or "method" for short, and at least includes the following steps S1 to S7.
[0042] S1: Get the training data for the GRU model and calculate the variance of the training data.
[0043] In one example, the training data is input into the GRU model one by one, and the training data is the input variable of the GRU model; the training data is a set of data with a maximum value and a minimum value, and the present application can calculate the variance of the training data according to the maximum value and the minimum value of the training data through the following relational formula 1. The variance is used to measure the degree of discreteness of the training data.
[0044] σ=(I max -I min ) 2 Relation 1
[0045] Wherein, σ represents the variance of the training data, and I max represents the maximum value of the training data, the I min Represents the minimum value of the training data.
[0046] S2: Perform Gaussian sampling according to the variance of the training data to obtain a set of Gaussian samples.
[0047] The so-called Gaussian sampling can be regarded as sampling according to the Gaussian distribution characteristics of the training data. Gaussian distribution, also known as normal distribution, is a form of probability distribution that is widely present in statistics and nature. Gaussian distribution describes a data distribution situation, that is, most of the data will be concentrated near the center, and the farther away from the center, the less data. This distribution form is reflected in many natural and social phenomena, such as human height, academic performance, physical quantities in natural phenomena, etc. This application samples the training data based on Gaussian distribution, so that the GRU model is derived and applied based on the normal distribution. Specifically, the probability density function of the Gaussian distribution is a bell-shaped curve, in which the mean determines the center position of the curve, and the variance σ determines the degree of discreteness of the curve. The peak of the curve appears at the mean, and then gradually decreases to both sides. The smaller the variance σ, the steeper the curve and the more concentrated the data distribution; the larger the variance σ, the flatter the curve and the more dispersed the data distribution. In practical applications, the probability distribution of many continuous random variables can be approximately regarded as a normal distribution, that is, the Gaussian distribution more realistically reflects the continuous distribution of samples. Therefore, this application performs GRU quantization and calculation based on the Gaussian distribution, which can produce high-precision mapping results for dense independent variable sets close to the mean, such as dense independent variable sets, thereby improving the accuracy of numerical calculations.
[0048] In an example, the present application may obtain the Gaussian sample set through the following equation 2:
[0049]
[0050] Wherein, the N(0,σ) represents a Gaussian distribution with a mean of zero and a variance of the calculated variance of the training data, that is, the N(0,σ) represents a Gaussian distribution with a mean of zero and a variance of σ. For the data set of the random variable training data, a mean of zero means that the average distribution of the data (values) in the training data is around the zero point. That is, if the data (value) is positive, there must be an equal number of data with negative values to balance it, so as to obtain a zero mean. The variance measures the square of the average deviation between the data in the training data and the mean, and the variance of σ means that the degree of fluctuation of the data in the training data relative to the mean is determined according to the variance adaptability of the input training data, and this degree of fluctuation is standardized to σ. Combining these two characteristics, a training data (i.e., a data set) with a mean of zero and a variance of σ can be regarded as a standardized distribution, in which the data is evenly distributed around the zero point, and the volatility or dispersion of the data is determined according to the variance adaptability of the input variable.
[0051] The d t~N(0,σ), t=1, 2, 3, ..., T represents non-repetitive sampling T times from the Gaussian distribution N(0,σ), and D is a set of Gaussian samples obtained by the T times of sampling.
[0052] S3: Sort the elements in the Gaussian sample set.
[0053] The elements are the data in the Gaussian sample set, that is, the samples obtained by the S2 sampling. By sorting the elements in the Gaussian sample set, the cumulative density function of the data can be better estimated, thereby more accurately analyzing the data features, which helps to improve the prediction accuracy of the GRU model.
[0054] In an example, the present application sorts the elements in the Gaussian sample set in ascending order, and the sorting is optionally performed through the following relationship 3:
[0055] Q=Sort(D)=[q t ], t=1, 2, 3, ..., T Relation 3
[0056] Wherein, Q represents the sorted Gaussian sample set, Sort(D) represents the sorting function used for D, and q t Represents the t-th element in the sorted Gaussian sample set.
[0057] In other examples, the elements in the Gaussian sample set may be sorted in other orders, such as from large to small. The order depends on the adaptability of the sorting function used and is not limited here.
[0058] S4: Reconstruct the domain of the activation function of the GRU model to limit the domain of the activation function of the GRU model to the sorted Gaussian sample set.
[0059] The activation function of the GRU model is used to add nonlinear characteristics to the GRU neural network, so that no matter how many layers the GRU neural network has, the output of any layer is a nonlinear combination of the input variables of that layer, and the input variables obtained by the next layer are also a nonlinear combination. This is equivalent to the GRU model being a multi-layer nonlinear model that can learn the characteristics of complex data such as images, audio, and text.
[0060] The role of the activation function is to help regulate the values of each layer flowing through the GRU neural network and limit it between the domains, thereby helping the GRU model to capture short-term dependencies in the time series. The domain of the existing activation function is correspondingly fixed, for example, the domain of the Tanh activation function is (-1, 1). The domain reconstruction of the present application is to limit the domain of the activation function of the GRU model to the Gaussian sample set, where the upper and lower limits of the domain of the activation function are set according to the variance adaptability of the training data, rather than fixed, and therefore can be applied to non-fixed input data.
[0061] In one example, the activation function of the GRU model can be at least one of the logical function Sigmoid, the hyperbolic tangent function Tanh, and the linear rectification function ReLU. The present application takes the GRU model including the Sigmoid activation function and the Tanh activation function as an example to describe the following steps. Correspondingly, the domain reconstruction of the activation function of the GRU model can be expressed as: the domains of the Sigmoid activation function and the Tanh activation function of the GRU model are limited to the sorted Gaussian sample set.
[0062] S5: Perform interpolation calculation on the activation function after domain reconstruction to obtain a corresponding range list.
[0063] In one example, the range list A of the Sigmoid activation function after domain reconstruction is calculated by equation 4, and the range list B of the Tanh activation function after domain reconstruction is calculated by equation 5:
[0064] A=Sigmoid(Q)=[Sigmoid(q t )], t = 1, 2, 3, ..., T Relation 4
[0065] B=Tanh(Q)=[Tanh(q t )], t = 1, 2, 3, ..., T Relational formula 5
[0066] S6: Map the elements in the range list to the fixed-point range list to complete the quantization of the GRU model.
[0067] The so-called quantization can be understood as converting the data of the GRU model (i.e., the data in the range list) from floating point representation to integer representation. The main goal is to reduce the GRU model parameters and model size, thereby reducing the storage and transmission overhead of the GRU model, while reducing the computational complexity and running time, while maintaining the performance and accuracy of the GRU model. In actual scenarios, the quantization can convert the GRU model to run on a specific hardware platform, so as to be suitable for image, audio, and text processing.
[0068] In one example, the present application maps the elements in the value range list to the fixed-point value range list through equation 6 to complete the quantization of the GRU model:
[0069]
[0070] The i represents the quantization factor, b is the quantization bit width, which is a known explanation, and A q Represents a fixed-point value range list mapped by the value range list A, the B q A fixed-point range list representing the mapping of range list B.
[0071] S7: Calculate the reset gate, update gate, candidate hidden state, and hidden state based on the quantized GRU model.
[0072] In addition to the aforementioned activation function, the network structure of the GRU model mainly includes input variables, gated hidden states, reset gates, update gates, and candidate hidden states. Among them, the reset gate helps capture short-term dependencies in the sequence; the update gate helps capture long-term dependencies in the sequence.
[0073] For a given time step t, the input variables are (n is the number of samples, d is the input feature dimension), the hidden state at the previous time step is (h is the number of hidden units), the reset gate R can be calculated by equation 7 t And according to equation 8, we can calculate the update gate Z t .
[0074] R t =σ(X t W xr +H t-1 W hr +b r ) Relation 7
[0075] Z t =σ(X t W xz +H t-1 W hz +b z ) Relation 8
[0076] in, is the weight parameter corresponding to the input variable, is the weight parameter corresponding to the hidden unit, is the bias parameter. Here, the Sigmoid activation function converts the input variable into the fixed-point value range list A mentioned above. q The corresponding value in .
[0077] Next, the candidate hidden state at time step t is calculated by the following relation 9:
[0078]
[0079] in, represents the candidate hidden state, and is the weight parameter corresponding to the input variable, is the bias parameter, and the symbol ⊙ is the product operator. Here, the Tanh activation function converts the input variable into the fixed-point value range list B mentioned above. q The corresponding value in ensures that the candidate hidden state The value is kept in the fixed-point range list B q between.
[0080] In the candidate hidden state In the example, the reset gate R is used t As coefficients to control which information in the hidden state of the previous time step is used when processing the input variables. Reset gate R t Input variable X when approaching the minimum value of the fixed-point range list t It dominates, indicating that the information contained in the input of the current time step is less relevant to the previous information; reset gate R t Input variable X when approaching the maximum value of the fixed-point range list t It is highly correlated with the previous long-term information and needs to be comprehensively considered to generate the information for the current time step.
[0081] Finally, the hidden state at time step t, i.e. the output, is calculated by the following equation 10:
[0082]
[0083] Among them, H t is a hidden state. Whenever the gate Z is updated t When it is close to the maximum value in the fixed-point value range list, the GRU model tends to keep only the old state. At this time, the information from the current time step t is basically ignored, effectively skipping the time step in the dependency chain. On the contrary, when the update gate Z t When it is close to the minimum value in the fixed-point value range list, the new hidden state H t will be close to the candidate hidden state
[0084] The aforementioned GRU quantization method can be applied to scenes such as images, audio, and text. The specific type of input training data and its maximum and minimum values can be determined according to the adaptability of the usage scenario.
[0085] As described above, in the method of the present application, Gaussian sampling is performed according to the variance of the training data to obtain a Gaussian sample set, and the domain of the activation function of the GRU model is limited to the Gaussian sample set, that is, the upper and lower limits of the domain of the activation function are adaptively set according to the variance of the training data, rather than fixed, and therefore can be applicable to non-fixed input data; and, the present application performs sampling based on the Gaussian distribution to achieve domain reconstruction of the activation function. Since the Gaussian distribution more realistically reflects the continuous distribution of samples, it can produce high-precision mapping results for dense independent variable sets close to the mean, for example, thereby improving the accuracy of numerical calculations; further, interpolation calculations are performed on the sample set obtained based on the Gaussian distribution, which is equivalent to calculating the value of the activation function based on a discrete empirical interpolation algorithm. The distance between the interpolation points reflects the true sample distribution, which is beneficial to improving the accuracy of numerical calculations.
[0086] The present application also provides a GRU quantization device, such as Figure 2 As shown, including:
[0087] An input module 21 is used to obtain training data input into the GRU model and calculate the variance of the training data;
[0088] A sampling module 22, configured to perform Gaussian sampling according to the variance of the training data to obtain a Gaussian sample set;
[0089] A sorting module 23, used for sorting the elements in the Gaussian sample set;
[0090] A reconstruction module 24, used for reconstructing the domain of the activation function of the GRU model to limit the domain of the activation function of the GRU model to the sorted Gaussian sample set;
[0091] A first calculation module 25, configured to perform interpolation calculation on the activation function after the domain is reconstructed to obtain a corresponding range list;
[0092] A mapping module 26, used for mapping the elements in the value range list to a fixed-point value range list to complete the quantization of the GRU model; and
[0093] The second calculation module 27 is used to calculate the reset gate, the update gate, the candidate hidden state and the hidden state based on the quantized GRU model.
[0094] The specific implementation methods of the above modules can be found in the above-mentioned embodiments of the GRU quantization method based on Gaussian distribution, which will not be described one by one here. Therefore, the GRU quantization device can also have the beneficial effects that can be achieved by any of the above-mentioned GRU quantization methods.
[0095] An embodiment of the present application also provides a storage medium having a program stored thereon, and when the program is executed by a processor, the steps of the Gaussian distribution-based GRU quantization method as in any example are implemented.
[0096] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0097] Since the instructions stored in the storage medium can execute the steps in any one of the GRU quantization methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the GRU quantization methods provided in the embodiments of the present application can be achieved. For details, please refer to the previous embodiments and will not be repeated here.
[0098] An embodiment of the present application also provides an intelligent device, including a memory and a processor, wherein a GRU quantization program is stored in the memory, and when the GRU quantization program is executed by the processor, the steps of the GRU quantization method described in any of the above examples are implemented; and / or, the intelligent device is set with a storage medium as in the above example, and the processor of the intelligent device loads the storage medium to execute the steps of the GRU quantization method, so as to achieve the beneficial effects that can be achieved by the GRU quantization method of the corresponding example of the present application.
[0099] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. For ordinary technicians in this field, all equivalent structural changes made using the contents of this specification and drawings are also included in the patent protection scope of the present application.
[0100] This document uses step codes such as S1, S2, etc., the purpose of which is to express the corresponding content more clearly and concisely, and does not constitute a substantial limitation on the order. When technical personnel in this field implement the specific steps, they may execute S2 first and then S1, etc., but these should all be within the scope of protection of this application.
[0101] Although the terms "first, second", etc. are used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. In addition, the singular forms "one", "an", and "the" are intended to include plural forms as well. The terms "or" and "and / or" are interpreted as inclusive, or mean any one or any combination. Only when the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way, an exception to this definition will occur.
Claims
1. A GRU quantization method based on Gaussian distribution, characterized in that: include: Obtain training data for input into the GRU model, and calculate the variance of the training data; Perform Gaussian sampling according to the variance of the training data to obtain a Gaussian sample set; Sorting the elements in the Gaussian sample set; Reconstructing the domain of the activation function of the GRU model to limit the domain of the activation function of the GRU model to the sorted Gaussian sample set; Performing interpolation calculation on the activation function after reconstructing the definition domain to obtain a corresponding value range list; Mapping the elements in the range list to a fixed-point range list to complete the quantization of the GRU model; Based on the quantized GRU model, the reset gate, update gate, candidate hidden states, and hidden states are calculated.
2. The method according to claim 1, characterized in that According to the maximum and minimum values of the training data, the variance of the training data is calculated by equation 1: σ=(I max -I min ) 2 Relation 1 The σ represents the variance of the training data, and the I max represents the maximum value of the training data, the I min Represents the minimum value of the training data.
3. The method according to claim 1 or 2, characterized in that: The Gaussian sample set is obtained through equation 2: The N(0,σ) represents a Gaussian distribution with a mean of zero and a variance equal to the calculated variance of the training data. t ~N(0,σ), t=1,2,3,..., T represents non-repeated sampling T times from the N(0,σ), and the D is a set of Gaussian samples obtained by the T times of sampling.
4. The method according to claim 1, characterized in that Sorting the elements in the Gaussian sample set includes: The elements in the Gaussian sample set are sorted in ascending order.
5. The method according to claim 4, characterized in that The elements in the Gaussian sample set are sorted by using equation 3: Q=Sort(D)=[q t ], t=1,2,3,...,T Relation 3 Q represents a sorted Gaussian sample set, Sort(D) represents a sorting function used for D, and q t represents the t-th element in the sorted Gaussian sample set.
6. The method according to claim 1 or 5, characterized in that: The domain of the activation function of the GRU model is limited to the sorted Gaussian sample set, including: The domains of the Sigmoid activation function and the Tanh activation function of the GRU model are limited to the sorted Gaussian sample set.
7. The method according to claim 6, characterized in that The activation function after the domain reconstruction is interpolated to obtain a corresponding range list, including: The range list A of the Sigmoid activation function after domain reconstruction is calculated by equation 4, and the range list B of the Tanh activation function after domain reconstruction is calculated by equation 5: A=Sigmoid(Q)=[Sigmoid(q t )], t=1,2,3,...,T Relation 4 B=Tanh(Q)=[Tanh(q t )], t=1,2,3,...,T Relationship 5.
8. The method according to claim 7, characterized in that The elements in the range list are mapped to the fixed-point range list through equation 6 to complete the quantization of the GRU model: The i represents the quantization factor, b is the quantization bit width, and A q represents the fixed-point value range list mapped by the value range list A, the B q Represents the fixed-point value range list mapped by the value range list B.
9. A storage medium, characterized in that: A program is stored, and when the program is executed by a processor, the steps of the Gaussian distribution-based GRU quantization method according to any one of claims 1 to 8 are implemented.
10. A smart device, characterized in that: A method comprising: comprising a processor capable of executing the steps of the Gaussian distribution-based GRU quantization method according to any one of claims 1 to 8; And / or, comprising the storage medium as claimed in claim 9.