Methods, devices, and electronic equipment for processing feature data of neural network models

Abstract

This application provides a method, apparatus, and electronic device for processing feature data of a neural network model. The method includes: acquiring feature data; performing fixed-point processing on the feature data to obtain first fixed-point data; scaling the first fixed-point data to obtain second fixed-point data; and performing approximate calculation on the second fixed-point data using a preset error function to obtain approximate result data; and integrating the approximate result data and the first fixed-point data to obtain nonlinear result data. This solution reduces hardware implementation complexity while improving the computational efficiency of model feature data, making it suitable for lightweight deployment in real-time scenarios.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, and electronic device for processing feature data of a neural network model. Background Technology

[0002] In neural network models, activation functions serve as the core module connecting linear transformations and nonlinear expressions. Their primary role is to introduce nonlinear characteristics, break the limitations of multi-layer linear transformations in the network, enable the model to fit complex nonlinear mapping relationships, and thus improve the model's ability to represent complex data and its fitting accuracy. They are an important component that determines the expressive power, training efficiency, and generalization performance of deep neural network models.

[0003] However, the calculation of activation functions in related technologies relies on floating-point operations. Floating-point operations not only require dedicated floating-point units in the hardware, which are difficult to map directly to fixed-point hardware, leading to increased hardware implementation complexity, but also involve complicated calculation steps and low data processing efficiency, which can affect the real-time performance of neural network model inference or training. Summary of the Invention

[0004] This application provides a method, apparatus, and electronic device for processing feature data of a neural network model. It solves the problem in related technologies that the calculation of activation functions leads to increased hardware implementation complexity and low data processing efficiency, which in turn affects the real-time performance of neural network model inference or training. It reduces hardware implementation complexity while improving the calculation efficiency of model feature data, making it suitable for lightweight deployment in real-time scenarios.

[0005] In a first aspect, embodiments of this application provide a feature data processing method for a neural network model, the method comprising:

[0006] Acquire feature data, and perform point-to-point processing on the feature data to obtain first point-to-point data; The first fixed-point data is scaled to obtain the second fixed-point data, and the second fixed-point data is approximated by matching a preset error function to obtain approximate result data. The approximate result data and the first fixed-point data are integrated to obtain the nonlinear result data.

[0007] Secondly, embodiments of this application also provide a feature data processing apparatus for a neural network model, comprising: The fixed-point processing module is configured to acquire feature data and perform fixed-point processing on the feature data to obtain first fixed-point data; The approximate calculation module is configured to scale the first fixed-point data to obtain the second fixed-point data, and to perform approximate calculation on the second fixed-point data by matching a preset error function to obtain approximate result data; The data integration module is configured to integrate the approximate result data and the first fixed-point data to obtain nonlinear result data.

[0008] Thirdly, embodiments of this application also provide an electronic device, the device comprising: One or more processors; Storage device, configured to store one or more programs, When one or more programs are executed by one or more processors, the one or more processors implement the feature data processing method of the neural network model of the present application embodiment.

[0009] Fourthly, embodiments of this application also provide a non-volatile storage medium for storing computer-executable instructions, which, when executed by a computer processor, are configured to perform a feature data processing method for a neural network model according to embodiments of this application.

[0010] In this embodiment, the feature data is processed into fixed-point data to obtain first fixed-point data. This eliminates the need for direct floating-point operations, enabling direct mapping between feature data and fixed-point hardware and reducing hardware resource consumption. The first fixed-point data is then scaled to obtain second fixed-point data, which adapts to the calculation range of the error function and improves the accuracy of approximate calculations. The second fixed-point data is then used to perform approximate calculations matching a preset error function to obtain approximate result data. This eliminates the need for transcendental function calculations, reducing hardware computational complexity. Finally, the approximate result data and the first fixed-point data are integrated to obtain nonlinear result data, which can replace the original activation function, introducing nonlinear characteristics into the neural network and ensuring the model's expressive power. This solution reduces hardware implementation complexity while improving the computational efficiency of model feature data, making it suitable for lightweight deployments in real-time scenarios. Attached Figure Description

[0011] Figure 1 A flowchart illustrating a feature data processing method for a neural network model provided in this application embodiment; Figure 2 A flowchart illustrating a specific implementation process for piecewise approximation calculation of a preset error function, provided in this application embodiment; Figure 3 A flowchart illustrating the specific implementation process of a first approximate calculation of a matching preset error function provided in this application embodiment; Figure 4 A flowchart illustrating the specific implementation process of a second approximation calculation for matching a preset error function, provided in this application embodiment; Figure 5 A flowchart illustrating a specific implementation process for obtaining second result data based on a second approximate mapping relationship, provided in this application embodiment; Figure 6 A flowchart illustrating a specific implementation process for constructing an index lookup table, provided in this application embodiment; Figure 7 A structural block diagram of a feature data processing device for a neural network model provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] The embodiments of this application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the embodiments of this application and are not intended to limit the scope of the embodiments. Furthermore, it should be noted that, for ease of description, only the parts relevant to the embodiments of this application are shown in the accompanying drawings, not the entire structure.

[0013] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0014] The feature data processing method for the neural network model provided in this application embodiment can be executed by a computer device. The computer device refers to any electronic device with data computing, processing and storage capabilities, such as mobile phones, PCs (Personal Computers), tablet computers and other terminal devices, or servers, etc. This application embodiment does not limit this.

[0015] Figure 1 A flowchart of a feature data processing method for a neural network model provided in this application embodiment is shown below. Figure 1 As shown, the feature data processing method of this neural network model specifically includes the following steps: Step S101: Obtain feature data and perform point-to-point processing on the feature data to obtain the first point data.

[0016] The feature data can be the input or output data of the forward propagation of each layer of the neural network model, specifically a set of numerical values ​​containing sample features (e.g., the output data of the feedforward network of a Transformer model), which requires the introduction of nonlinear characteristics. Acquiring the feature data can be a hardware operation that reads the feature data from the registers or storage units of the neural network hardware computing unit, supporting pipelined data reading. Since the feature data is floating-point data, it can be converted to a fixed-point format through fixed-point conversion. Specifically, the fixed-point format is Qm.n, where m is the integer bit width including the sign bit, and n is the fractional bit width. For example, if the total width is 32 bits, m is 16 bits, and n is 16 bits. This fixed-point format can be adapted to hardware computing operations, facilitating subsequent calculations by hardware logic circuits (e.g., adders, multipliers, shifters, etc.).

[0017] Step S102: Scale the first fixed-point data to obtain the second fixed-point data, and perform approximate calculations on the second fixed-point data using a preset error function to obtain approximate result data.

[0018] The scaling process can be adapted to the calculation range of a preset error function, performing fixed-point multiplication on the fixed-point data. For example, multiplying the first fixed-point data by a preset fixed-point constant yields the second fixed-point data. The preset error function can be a Gaussian error function, a complementary error function, etc., and is not limited in this application. The approximation calculation can use numerical approximation methods such as polynomial or exponential fitting to replace the direct calculation of the error function. This ensures that the calculation process of the second fixed-point data is compatible with the mathematical characteristics of the preset error function, while retaining only the basic operations that can be implemented by the hardware (such as addition, multiplication, and shifting). The approximate result data obtained through the approximation calculation can be regarded as a high-precision approximation result of the error function, without the involvement of floating-point operations.

[0019] In one embodiment, a specific implementation process for piecewise approximation calculation of a preset error function is described. Please refer to [reference needed]. Figure 2 This is a flowchart illustrating the specific implementation process of a piecewise approximation calculation of a preset error function provided in an embodiment of this application, as shown below. Figure 2 As shown, the specific implementation steps for obtaining approximate result data by matching the second fixed-point data with a preset error function are as follows: Step S1021: When the absolute value of the second fixed-point data is less than or equal to the first preset threshold, the second fixed-point data is subjected to a first approximate calculation of the matching preset error function to obtain the first result data.

[0020] In one embodiment, a specific implementation process for a first approximation calculation matching a preset error function is described. Please refer to [reference needed]. Figure 3This is a flowchart illustrating the specific implementation process of a first approximate calculation of a matching preset error function provided in an embodiment of this application, as shown below. Figure 3 As shown, the specific implementation steps for obtaining the first result data by performing a first approximation calculation using a preset error function to match the second fixed-point data are as follows: Step S10211: Perform polynomial fitting on the preset error function to obtain the first approximate mapping relationship.

[0021] Specifically, polynomial fitting can involve constructing a finite-order polynomial so that its function value approximates the true value of the preset error function within a small amplitude range. The specific polynomial expression can be obtained by fitting the preset error function in the offline stage, based on the actual application. For example, the preset error function used is a Gaussian error function. Correspondingly, the first approximate mapping relationship can be ,in, For about The polynomial expression can be obtained by determining the polynomial coefficients offline, for example, by solving the polynomial coefficients using algorithms such as least quadratic multiplication, thus minimizing the residual between the polynomial and the error function. These polynomial coefficients can be quantized and stored in a fixed-point format. Optionally, the polynomial computation can employ Horner expansion, involving only fixed-point multiplication and addition operations, making it suitable for hardware pipelining.

[0022] Step S10212: Calculate the first result data based on the second fixed-point data and the first approximate mapping relationship.

[0023] After obtaining the first approximate mapping relationship, the second fixed-point data can be substituted into the polynomial corresponding to the first approximate mapping relationship as the input basis for hardware calculation.

[0024] As described above, by fitting the preset error function with a polynomial, only fixed-point multiplication, addition, and shift operations can be used, reducing hardware resource consumption and computational latency. All calculations are implemented by basic hardware circuits, without iteration or complex function calls.

[0025] Step S1022: If the absolute value of the second fixed-point data is greater than the first preset threshold, perform a second approximate calculation on the second fixed-point data to match the preset error function to obtain the second result data.

[0026] Steps S1021 and S1022 can be executed in parallel or sequentially, and this application does not impose any limitation on them. The first preset threshold can be a critical value determined based on the numerical change characteristics of a preset error function, and it serves as the basis for dividing the intervals for the piecewise approximation of the error function.

[0027] Optionally, the process for determining the first preset threshold is as follows: The slope characteristics of the function are obtained by analyzing the rate of change of the preset error function; the critical threshold of change of the preset error function is determined based on the slope characteristics and the preset accuracy threshold, and the critical threshold of change is determined as the first preset threshold.

[0028] One method for analyzing the rate of change of a preset error function is to perform derivative analysis on the preset error function. For example, calculating the first derivative of the error function; the absolute value of the derivative is the slope of the function, reflecting how quickly the function value changes. The slope characteristic of the function can be the slope variation pattern of the error function over the entire interval, for example, the Gaussian error function. The slope is large in the small amplitude range and small in the large amplitude range. The preset accuracy threshold can be the maximum allowable error of the neural network model's approximation of the activation function, determined by the accuracy requirements of the neural network model. Specifically, a Gaussian error function is used. The functional characteristics can be used to determine the Gaussian error function. The slope decays rapidly and slowly, thus locating the end of the rapid decay phase. This end is defined as the initial candidate range for the threshold, ensuring that the values ​​within the candidate range satisfy the balance point of different approximation methods. Finally, a critical threshold that meets a preset accuracy threshold can be selected from the candidate range to simultaneously satisfy the maximum allowable absolute error requirements of different approximation methods. This critical threshold can be determined as the first preset threshold.

[0029] Specifically, if the absolute value of the second fixed-point data is less than or equal to the first preset threshold, it can be considered that the second fixed-point data is in a range sensitive to changes in the preset error function. The curve in this range is steep, and the function's rate of change is large, meaning the value changes significantly with small changes in the input, requiring high accuracy in the approximation calculation. For example, expanding the aforementioned range into a polynomial form using Taylor expansion allows for accurate approximation of its true value. Furthermore, polynomial calculations can be expanded using a Horner structure, requiring only fixed-point multiplication and addition, eliminating the need for complex hardware units and significantly reducing logic resource consumption. The fitting formula can be adapted to the activation function to be replaced. If the absolute value of the second fixed-point data is greater than the first preset threshold, it can be considered that the second fixed-point data is in a range where the preset error function's value changes smoothly. The curve in this range tends to be flat, and the function's rate of change is small, meaning its value hardly changes with large changes in the input, requiring relatively lower accuracy in the approximation calculation. For example, an exponential function approximation can be used, which better reflects its mathematical characteristics. The specific calculation formula can be adapted to the activation function to be replaced.

[0030] Therefore, the preset error function is piecewise fitted to accurately match the interval characteristics of the error function, balancing accuracy and hardware complexity, and reducing the difficulty of hardware implementation.

[0031] In one embodiment, a specific implementation process for a second approximation calculation matching a preset error function is described. Please refer to [reference needed]. Figure 4 This is a flowchart illustrating the specific implementation process of a second approximation calculation for matching a preset error function, as provided in an embodiment of this application. Figure 4 As shown, the specific implementation steps for obtaining the second result data by performing a second approximation calculation using a preset error function to match the second fixed-point data are as follows: Step S10221: Perform exponential fitting on the preset error function to obtain the second approximate mapping relationship.

[0032] Exponential fitting involves constructing an exponential function to approximate the true value of a preset error function within its large amplitude range. The specific polynomial expression can be obtained by fitting the preset error function during the offline stage. For example, the preset error function used is a Gaussian error function. Correspondingly, the second approximate mapping relationship can be ,in, Since it is a low-order polynomial, it can be calculated using only fixed-point multiplication and addition operations. It is an exponential function. It should be noted that the Gaussian error function... Direct calculation involves multiple layers of transcendental functions, while exponential fitting can transform it into a single exponential function calculation.

[0033] In one embodiment, the second approximate mapping relationship corresponds to an exponential lookup table. This exponential lookup table can be built offline and stores fixed-point values ​​of the exponential function. The exponential function calculation logic in the second approximate mapping relationship is hardware-associated with the exponential lookup table, allowing the exponential function calculation to directly call the lookup table data, rather than being calculated in real time. Please refer to [reference needed]. Figure 5 This is a flowchart illustrating a specific implementation process for obtaining second result data based on a second approximate mapping relationship, as provided in this application embodiment. Figure 5 As shown, the specific implementation steps for calculating the second result data based on the second fixed-point data and the second approximate mapping relationship are as follows: Step S102211: Determine whether the second fixed-point data exceeds the range of the lookup table corresponding to the exponential lookup table.

[0034] The lookup table range can be the input range of the exponential function covered by the exponential lookup table, which is a preset numerical range during offline construction. The exponential function value changes very little outside this range.

[0035] Step S102212: If the second fixed-point data exceeds the range of the lookup table, determine the second result data based on the boundary values ​​of the exponential lookup table.

[0036] If the second fixed-point data exceeds the lookup table range, the second result data can be directly determined based on the boundary values. For example, if it's below the lower boundary, the lower boundary value can be directly used as the second result data. Similarly, if it's above the upper boundary, the upper boundary value can be directly used as the second result data. Since the exponential function's value changes very little outside the lookup table range, high-precision calculations such as interpolation are unnecessary. Pre-judging and filtering out-of-range data reduces unnecessary hardware operations.

[0037] Step S102213: If the second fixed-point data does not exceed the range of the lookup table, calculate the lookup table index and interpolation ratio coefficient based on the second fixed-point data.

[0038] If the second fixed-point data does not exceed the lookup table range, the lookup table index and interpolation ratio can be calculated based on the second fixed-point data. The lookup table index can be an integer address value used to access the index lookup table, determined by the integer part or high-order part of the second fixed-point data, and is a unique identifier for hardware reading of the lookup table data. The interpolation ratio can be a fixed-point number determined by the fractional part of the second fixed-point data, used for linear interpolation calculations. Specifically, based on fixed-point operations implemented by hardware shifters and adders, the second fixed-point data can be decomposed into "index + fractional part," with the fractional part being the interpolation ratio.

[0039] Step S102214: Read the two adjacent fixed-point result values ​​corresponding to the index of the lookup table from the index lookup table.

[0040] Among them, two adjacent fixed-point result values ​​can be the fixed-point value corresponding to the lookup table index and the fixed-point value corresponding to the next index, which are the two basic data of linear interpolation.

[0041] Step S102215: The second result data is obtained by weighted calculation based on the two fixed-point result values ​​and the interpolation ratio coefficient.

[0042] Weighted calculation can be a process of performing fixed-point multiplication and addition operations on two fixed-point result values ​​and interpolation scaling factors according to a linear interpolation formula, implemented by hardware multipliers and adders. For example, the relevant formulas for weighted calculation are as follows: ,in, For the second result data, These are linear interpolation coefficients. To find the fixed-point value corresponding to the table index, This is the fixed-point value corresponding to the next index. It should be noted that steps 102212 and steps 102213-102214 can be executed serially or in parallel, and this application does not impose any restrictions on them.

[0043] Therefore, fixed-point data can be quickly read based on the exponential lookup table, replacing the complex calculation of the exponential function and improving computational efficiency. Furthermore, linear interpolation can compensate for the quantization error of the lookup operation and improve the approximate accuracy of the exponential function.

[0044] Step S10222: Calculate the second result data based on the second fixed-point data and the second approximate mapping relationship.

[0045] Once the second approximate mapping relationship is obtained, the second fixed-point data can be substituted into the polynomial corresponding to the second approximate mapping relationship as the input basis for hardware calculation.

[0046] Step S103: Integrate the approximate result data and the first fixed-point data to obtain the nonlinear result data.

[0047] After obtaining the approximate result data and feature data, the approximate result data and feature data can be combined using fixed-point multiplication and addition calculations according to the set function calculation relationship to obtain the nonlinear result data. It should be noted that steps S101-S103 can replace the original activation function calculation process in the neural network model, introducing nonlinear characteristics to the neural network while adapting to data processing in fixed-point hardware. Taking the activation function GELU as an example, its calculation formula is as follows:

[0048] Accordingly, the activation function GELU can be decomposed and calculated in steps S101-S103 to map it to fixed-point hardware for computation. The feature data is then processed by fixed-point scaling to obtain the first fixed-point data. This first fixed-point data is then scaled (by a preset fixed-point constant). (Multiplication) yields the second fixed-point data, with the preset error function being the Gaussian error function. Matching the second fixed-point data The approximate calculation yields approximate result data. Finally, referring to the original calculation relationship of the activation function GELU, after obtaining the approximate result data and the first fixed-point data, the approximate result data and the first fixed-point data can be integrated to obtain the nonlinear result data, i.e. Of course, the above is only an example calculation process for the activation function GELU. The specific implementation steps can be adapted by combining other activation functions, which are not limited here.

[0049] The above-described method involves performing fixed-point processing on the feature data to obtain the first fixed-point data. This eliminates the need for direct floating-point operations, enabling a direct mapping between the feature data and fixed-point hardware and reducing hardware resource consumption. Scaling the first fixed-point data to obtain the second fixed-point data adapts to the calculation range of the error function, improving the accuracy of approximate calculations. Approximate calculations using the second fixed-point data and a preset error function yield approximate result data, eliminating the need for transcendental function calculations and reducing hardware computational complexity. Integrating the approximate result data with the first fixed-point data yields nonlinear result data, which can replace the original activation function, introducing nonlinear characteristics into the neural network and ensuring the model's expressive power. This approach reduces hardware implementation complexity while improving the computational efficiency of model feature data, making it suitable for lightweight, real-time deployments.

[0050] In one embodiment, Figure 6 A flowchart illustrating a specific implementation process for constructing an index lookup table, as provided in this application embodiment, is shown below. Figure 6 As shown, the specific implementation steps for constructing the index lookup table are as follows: Step S601: Determine multiple lookup table indices corresponding to the preset exponential function and the sampling points corresponding to each lookup table index based on the preset interval and lookup table depth.

[0051] The preset interval can be the input range of the exponential function when building the exponential lookup table offline, determined by the numerical characteristics of the exponential function. The lookup depth can be the number of items in the exponential lookup table, i.e., the number of "index-fixed-point value" key-value pairs stored in the lookup table. A larger lookup depth results in higher approximate accuracy but also higher hardware resource consumption. This preset interval and lookup depth serve as the basis for dividing the lookup table index and sampling points, ensuring that the sampling points uniformly cover the preset interval. The step size can be calculated according to the preset interval and lookup depth, uniformly dividing the sampling points and assigning a unique lookup table index to each sampling point. For example, the preset interval is... The lookup table depth is N, and the step size can be... The sampling points can be the input values ​​of the exponential function, evenly divided within a preset interval according to the lookup table depth, serving as the input data for offline calculation of the exponential function values. For example, ,in, For indexing, Step size, This is the minimum value within the preset interval.

[0052] Step S602: Calculate the exponential function value corresponding to each sampling point, and perform fixed-point processing on each exponential function value to obtain the fixed-point result value corresponding to each exponential function value.

[0053] The calculation of the exponential function value corresponding to the sampling point can be performed in an offline high-precision floating-point environment, calculating the true value of the exponential function for each sampling point. The fixed-point result value obtained after converting the exponential function value of the sampling point to a fixed-point value is the core data stored in the exponential lookup table.

[0054] Step S603: Construct an index lookup table based on each lookup table index and the corresponding fixed-point result value.

[0055] Each lookup table index and its corresponding fixed-point result value can be stored as key-value pairs for easy retrieval in subsequent calculations.

[0056] As mentioned above, the sampling points uniformly cover the preset interval, ensuring that the approximate accuracy of the exponential function is uniform throughout the entire interval without any local precision loss. Furthermore, the floating-point exponential function values ​​are converted into the same fixed-point format as the feature data, allowing the hardware to directly perform multiplication-addition or interpolation calculations after reading them without the need for format conversion, thus improving computational efficiency.

[0057] Figure 7 This is a structural block diagram of a feature data processing device for a neural network model provided in an embodiment of this application. The device is configured to execute the feature data processing method for the neural network model provided in the above embodiment, and has corresponding functional modules and beneficial effects for executing the method. Figure 7 As shown, the device specifically includes: The fixed-point processing module 701 is configured to acquire feature data and perform fixed-point processing on the feature data to obtain the first fixed-point data; The approximate calculation module 702 is configured to scale the first fixed-point data to obtain the second fixed-point data, and to perform approximate calculation on the second fixed-point data by matching a preset error function to obtain approximate result data; The data integration module 703 is configured to integrate the approximate result data and the first fixed-point data to obtain the nonlinear result data.

[0058] The above-described method involves performing fixed-point processing on the feature data to obtain the first fixed-point data. This eliminates the need for direct floating-point operations, enabling a direct mapping between the feature data and fixed-point hardware and reducing hardware resource consumption. Scaling the first fixed-point data to obtain the second fixed-point data adapts to the calculation range of the error function, improving the accuracy of approximate calculations. Approximate calculations using the second fixed-point data and a preset error function yield approximate result data, eliminating the need for transcendental function calculations and reducing hardware computational complexity. Integrating the approximate result data with the first fixed-point data yields nonlinear result data, which can replace the original activation function, introducing nonlinear characteristics into the neural network and ensuring the model's expressive power. This approach reduces hardware implementation complexity while improving the computational efficiency of model feature data, making it suitable for lightweight, real-time deployments.

[0059] In one possible embodiment, the approximation calculation module 702 is specifically configured as follows: If the absolute value of the second fixed-point data is less than or equal to the first preset threshold, the second fixed-point data is subjected to a first approximate calculation of the matching preset error function to obtain the first result data. If the absolute value of the second fixed-point data is greater than the first preset threshold, the second fixed-point data is subjected to a second approximation calculation matching the preset error function to obtain the second result data.

[0060] In one possible embodiment, the approximation calculation module 702 is further configured as follows: The first approximate mapping relationship is obtained by polynomial fitting of the preset error function; The first result data is obtained by calculating based on the second fixed-point data and the first approximate mapping relationship.

[0061] In one possible embodiment, the approximation calculation module 702 is further configured as follows: The second approximate mapping relationship is obtained by performing exponential fitting on the preset error function; The second result data is obtained by calculating based on the second fixed-point data and the second approximate mapping relationship.

[0062] In one possible embodiment, an exponential lookup table is provided for the second approximate mapping relationship; The approximate calculation module 702 is further configured as follows: Determine whether the second fixed-point data exceeds the range of the lookup table corresponding to the index lookup table; If the second fixed-point data exceeds the range of the lookup table, the second result data is determined based on the boundary values ​​of the exponential lookup table.

[0063] In one possible embodiment, the approximation calculation module 702 is further configured as follows: If the second fixed-point data does not exceed the range of the lookup table, calculate the lookup table index and interpolation ratio coefficient based on the second fixed-point data; Read the two adjacent fixed-point result values ​​corresponding to the index in the index lookup table; The second result data is obtained by weighting the two fixed-point result values ​​and the interpolation ratio coefficient.

[0064] In one possible embodiment, the process of constructing the index lookup table is as follows: Based on the preset interval and the lookup table depth, determine multiple lookup table indices corresponding to the preset exponential function and the sampling points corresponding to each lookup table index; Calculate the exponential function value corresponding to each sampling point, and perform fixed-point processing on each exponential function value to obtain the fixed-point result value corresponding to each exponential function value; An index lookup table is constructed based on each lookup table index and its corresponding fixed-point result value.

[0065] In one possible embodiment, the process for determining the first preset threshold is as follows: The slope characteristics of the function are obtained by analyzing the rate of change of the preset error function. The critical threshold for the change of the preset error function is determined based on the slope characteristics of the function and the preset accuracy threshold, and the critical threshold for the change is set as the first preset threshold.

[0066] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 8 As shown, the device includes a processor 801, a memory 802, an input device 803, and an output device 804; the number of processors 801 in the device can be one or more. Figure 8 Taking a processor 801 as an example; the processor 801, memory 802, input device 803, and output device 804 in the device can be connected via a bus or other means. Figure 8 Taking a bus connection as an example, the memory 802, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the feature data processing method of the neural network model in this embodiment. The processor 801 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 802, thereby implementing the aforementioned feature data processing method of the neural network model. The input device 803 can be configured to receive input digital or character information and generate key signal inputs related to user settings and function control of the device. The output device 804 may include a display screen or other display device.

[0067] The electronic device provided above can be used to execute the feature data processing method of the neural network model provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0068] This application also provides a non-volatile storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are configured to perform a feature data processing method for a neural network model described in the above embodiments. The method includes: acquiring feature data; performing fixed-point processing on the feature data to obtain first fixed-point data; scaling the first fixed-point data to obtain second fixed-point data; performing approximate calculation on the second fixed-point data by matching a preset error function to obtain approximate result data; and integrating the approximate result data and the first fixed-point data to obtain nonlinear result data.

[0069] Storage medium – any type of memory device or storage device. The term “storage medium” is intended to include: mounting media, such as CD-ROM, floppy disk, or magnetic tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media, or optical storage; registers or other similar types of memory elements, etc. Storage medium may also include other types of memory or combinations thereof. Furthermore, storage medium may reside in a first computer system in which the program is executed, or it may reside in a different second computer system connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term “storage medium” may include two or more storage media residing in different locations (e.g., in different computer systems connected via a network). Storage medium may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.

[0070] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the feature data processing method of the neural network model as described above, but can also execute related operations in the feature data processing method of the neural network model provided in any embodiment of this application.

[0071] It is worth noting that in the embodiments of the feature data processing device of the above neural network model, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not configured to limit the protection scope of the embodiments of this application.

[0072] It should be noted that the numbering of each step in this solution is only used to describe the overall design framework of this solution and does not indicate a necessary sequential relationship between the steps. As long as the overall implementation process conforms to the overall design framework of this solution, it falls within the protection scope of this solution. The order of the text in the description is not an exclusive limitation on the specific implementation process of this solution. Those skilled in the art should understand that the embodiments of this application can be provided as methods, systems, or computer program products. In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-persistent memory in computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0073] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0074] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for processing feature data of a neural network model, characterized in that, The method includes: Acquire feature data, and perform point-to-point processing on the feature data to obtain first point-to-point data; The first fixed-point data is scaled to obtain the second fixed-point data, and the second fixed-point data is approximated by matching a preset error function to obtain approximate result data. The approximate result data and the first fixed-point data are integrated to obtain the nonlinear result data.

2. The feature data processing method for a neural network model according to claim 1, characterized in that, The approximate result data obtained by performing an approximate calculation on the second fixed-point data using a preset error function includes: If the absolute value of the second fixed-point data is less than or equal to the first preset threshold, the second fixed-point data is subjected to a first approximate calculation matching a preset error function to obtain the first result data. If the absolute value of the second fixed-point data is greater than the first preset threshold, the second fixed-point data is subjected to a second approximation calculation matching a preset error function to obtain the second result data.

3. The feature data processing method for a neural network model according to claim 2, characterized in that, The first approximate calculation of the second fixed-point data by matching a preset error function to obtain the first result data includes: The first approximate mapping relationship is obtained by performing polynomial fitting on the preset error function; The first result data is obtained by calculating based on the second fixed-point data and the first approximate mapping relationship.

4. The feature data processing method for a neural network model according to claim 2, characterized in that, The second approximation calculation of the second fixed-point data by matching a preset error function to obtain the second result data includes: The second approximate mapping relationship is obtained by performing exponential fitting on the preset error function; The second result data is obtained by calculating based on the second fixed-point data and the second approximate mapping relationship.

5. The feature data processing method for a neural network model according to claim 4, characterized in that, The second approximate mapping relationship is provided with an exponential lookup table; The step of calculating the second result data based on the second fixed-point data and the second approximate mapping relationship includes: Determine whether the second fixed-point data exceeds the range of the lookup table corresponding to the index lookup table; If the second fixed-point data exceeds the range of the lookup table, the second result data is determined based on the boundary values ​​of the exponential lookup table.

6. The feature data processing method for a neural network model according to claim 5, characterized in that, After determining whether the second fixed-point data exceeds the range of the lookup table corresponding to the index lookup table, the method further includes: If the second fixed-point data does not exceed the range of the lookup table, calculate the lookup table index and interpolation ratio coefficient based on the second fixed-point data; Read the two adjacent fixed-point result values ​​corresponding to the index of the index from the index lookup table; The second result data is obtained by weighting the two fixed-point result values ​​and the interpolation ratio coefficient.

7. The feature data processing method for a neural network model according to claim 5 or 6, characterized in that, The process of constructing the index lookup table is as follows: Based on the preset interval and the lookup table depth, determine multiple lookup table indices corresponding to the preset exponential function and the sampling points corresponding to each lookup table index; Calculate the exponential function value corresponding to each sampling point, and perform fixed-point processing on each exponential function value to obtain the fixed-point result value corresponding to each exponential function value; An index lookup table is constructed based on each of the lookup table indexes and the corresponding fixed-point result values.

8. The feature data processing method for a neural network model according to claim 2, characterized in that, The process for determining the first preset threshold is as follows: The slope characteristics of the function are obtained by analyzing the rate of change of the preset error function; The change threshold of the preset error function is determined based on the slope characteristics of the function and the preset accuracy threshold, and the change threshold is determined as the first preset threshold.

9. A feature data processing device for a neural network model, characterized in that, include: The fixed-point processing module is configured to acquire feature data and perform fixed-point processing on the feature data to obtain first fixed-point data; The approximate calculation module is configured to scale the first fixed-point data to obtain the second fixed-point data, and to perform approximate calculation on the second fixed-point data by matching a preset error function to obtain approximate result data; The data integration module is configured to integrate the approximate result data and the first fixed-point data to obtain nonlinear result data.

10. An electronic device, the electronic device comprising: One or more processors; A memory configured to store one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the feature data processing method for the neural network model according to any one of claims 1-8.

11. A non-volatile storage medium storing computer-executable instructions, which, when executed by a computer processor, are configured to perform a feature data processing method for a neural network model according to any one of claims 1-8.

Images