Data processing methods and data processing devices

Abstract

This application discloses a data processing method and apparatus in the field of artificial intelligence, particularly in the field of model compression. The method includes: performing target processing on input data using a trained neural network to obtain a target result; the target processing includes: performing a first processing on the input data using the neural network to obtain a first fixed-point number; determining a second fixed-point number corresponding to the first fixed-point number according to a fixed-point number correspondence; obtaining the target result based on the second fixed-point number; the second fixed-point number is equal to a third fixed-point number obtained by performing a second processing on the first fixed-point number, the second processing including: dequantizing the first fixed-point number to obtain a first floating-point number, processing the first floating-point number using a target nonlinear activation function to obtain a second floating-point number, quantizing the second floating-point number to obtain the third fixed-point number; and outputting the target result; which can reduce computational complexity and reduce the storage space occupied.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more particularly to a data processing method and a data processing apparatus. Background Technology

[0002] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. Research in the field of AI includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and fundamental AI theories.

[0003] Currently, neural networks, such as recurrent neural networks (RNNs), gated recurrent units (GRUs), long short-term memory (LSTM), bidirectional long short-term memory (BiLSTM), and convolutional neural networks (CNNs), are increasingly widely used on mobile devices (e.g., smartphones, tablets). For example, terminals use RNNs to handle time-related processing tasks, such as automatic speech recognition (ASR) and semantic understanding. However, due to the limited computing power of the terminal, the use of neural networks for floating-point calculations to perform inference tasks results in slow inference speeds. Therefore, there is a need to research data processing methods with higher computational efficiency. Summary of the Invention

[0004] This application discloses a data processing method and a data processing apparatus that can reduce computational complexity, improve model inference speed, and reduce the storage space occupied.

[0005] In a first aspect, embodiments of this application provide a data processing method, the method comprising: performing target processing on input data using a trained neural network to obtain a target result; the input data including multiple computer-processable signals, the target processing comprising: performing a first processing on the input data using the neural network to obtain a first fixed-point number; determining a second fixed-point number corresponding to the first fixed-point number according to a fixed-point number correspondence; obtaining the target result based on the second fixed-point number; the fixed-point number correspondence including a correspondence between the first fixed-point number and the second fixed-point number, the second fixed-point number being equal to a third fixed-point number obtained by performing a second processing on the first fixed-point number; the second processing comprising: dequantizing the first fixed-point number to obtain a first floating-point number; processing the first floating-point number using a target nonlinear activation function to obtain a second floating-point number; quantizing the second floating-point number to obtain the third fixed-point number; the target nonlinear activation function being the activation function used by the neural network; and outputting the target result.

[0006] The execution subject of this application embodiment is a data processing device. The technical problem solved by the data processing device performing the operation of determining the second fixed-point number corresponding to the first fixed-point number based on the fixed-point number correspondence is the same as the technical problem solved by performing a second processing on the first fixed-point number to obtain the second fixed-point number; both result in obtaining the second fixed-point number. In other words, the data processing device achieves the purpose that could only be achieved by performing a second processing on the first fixed-point number by performing the operation of determining the second fixed-point number corresponding to the first fixed-point number based on the fixed-point number correspondence, without needing to perform the operation of "dequantizing the first fixed-point number to obtain a first floating-point number, processing the first floating-point number using a target nonlinear activation function to obtain a second floating-point number, and quantizing the second floating-point number to obtain the third fixed-point number." It should be understood that the computational complexity of performing "dequantizing the first fixed-point number to obtain a first floating-point number, processing the first floating-point number using a target nonlinear activation function to obtain a second floating-point number, and quantizing the second floating-point number to obtain the third fixed-point number" is higher than the computational complexity of performing "determining the second fixed-point number corresponding to the first fixed-point number based on the fixed-point number correspondence." Therefore, determining the second fixed-point number corresponding to the first fixed-point number based on the fixed-point number correspondence reduces computational complexity and improves model inference speed. Furthermore, the data processing device's implementation of "performing a second process on the first fixed-point number" requires floating-point operations, consuming significant storage space. Since the data processing device uses fixed-point operations throughout the entire target processing flow, the storage space required can be reduced.

[0007] In this embodiment of the application, the second fixed-point number corresponding to the first fixed-point number is determined according to the fixed-point number correspondence; this can reduce computational complexity, improve model inference speed, and reduce the storage space occupied.

[0008] In one possible implementation, determining the second fixed-point number corresponding to the first fixed-point number based on the fixed-point number correspondence includes: searching the fixed-point number correspondence table for the second fixed-point number corresponding to the first fixed-point number; the fixed-point number correspondence table includes the fixed-point number correspondence.

[0009] The data processing device can pre-store the lookup table of fixed-point correspondences. In this implementation, by looking up the lookup table, the second fixed-point number corresponding to the first fixed-point number can be obtained accurately and quickly.

[0010] In one possible implementation, the first processing includes performing matrix multiplication on the input data using weight data. The first processing does not include quantization or dequantization operations. The input data includes fixed-point numbers, and the weight data includes fixed-point numbers.

[0011] In this implementation, the data processing device uses fixed-point arithmetic during the first processing, which has low computational complexity and consumes less storage resources.

[0012] In one possible implementation, obtaining the target result based on the second fixed-point number includes: performing a first normalization process on the second fixed-point number to obtain a fourth fixed-point number; the first normalization process includes: calculating the value of a first formula to obtain a third floating-point number, wherein the parameters in the first formula include floating-point numbers obtained by data type conversion of the second fixed-point number; quantizing the third floating-point number to obtain a fifth fixed-point number; calculating the sum of the fifth fixed-point number and the sixth fixed-point number to obtain the fourth fixed-point number; the first formula is a part of a first normalization formula corresponding to the first normalization process, and the sixth fixed-point number is an offset value in the first normalization formula; and obtaining the target result based on the fourth fixed-point number.

[0013] In the process of performing the first normalization process on the second fixed-point number to obtain the fourth fixed-point number, only one quantization operation is required, and no dequantization operation is needed, resulting in low computational complexity. In other words, in the process of normalizing one fixed-point number to obtain another fixed-point number, only one quantization operation is required.

[0014] In this implementation, the second fixed-point number is normalized by a first normalization formula obtained through mathematical transformation and merging of the normalization formula and the inverse quantization formula, retaining only one necessary quantization operation, which can reduce the computational complexity of the normalization operation and improve the computational efficiency.

[0015] In one possible implementation, the target nonlinear activation function is obtained by multiplying the scaling factor in the first normalization formula by the initial nonlinear activation function.

[0016] In this implementation, by incorporating the scaling factor in the first normalization formula into the nonlinear activation function, the computational cost of the normalization operation can be reduced.

[0017] In one possible implementation, obtaining the target result based on the second fixed-point number includes: performing a second normalization process on the second fixed-point number to obtain a seventh fixed-point number; the second normalization process includes: calculating the value of a second normalization formula to obtain a fourth floating-point number, wherein the parameters in the second normalization formula include a floating-point number obtained by data type conversion of the second fixed-point number; quantizing the fourth floating-point number to obtain the seventh fixed-point number; and obtaining the target result based on the seventh fixed-point number.

[0018] In this implementation, the second fixed-point number is normalized by a second normalization formula obtained by mathematical transformation and merging of the normalization formula and the inverse quantization formula. Only one necessary quantization operation is retained, which can reduce the computational complexity of the normalization operation and improve the computational efficiency.

[0019] In one possible implementation, the target nonlinear activation function is obtained by multiplying the scaling factor in the second normalization formula by the initial nonlinear activation function.

[0020] In this implementation, by incorporating the scaling factor in the first normalization formula into the nonlinear activation function, the computational cost of the normalization operation can be reduced.

[0021] In one possible implementation, the neural network is any one of a recurrent neural network (RNN), a gated recurrent unit (GRU), a long short-term memory (LSTM), a bidirectional long short-term memory (BiLSTM), a simple recurrent unit (SRU), and future RNN variants.

[0022] The neural network can be a neural network model used to process time-related features. It is understood that the neural network summarized in this application can be an RNN, a variant of any current RNN, or a future variant of an RNN.

[0023] In one possible implementation, the plurality of computer-processable signals include at least one of: voice signals, text signals, or image signals.

[0024] In one possible implementation, the target nonlinear activation function is an activation function with upper and lower bounds, or the target nonlinear activation function is a family of ReLU functions with an upper bound.

[0025] Secondly, embodiments of this application provide a data processing apparatus, comprising: a processing module, configured to perform target processing on input data using a trained neural network to obtain a target result; the input data includes multiple computer-processable signals, the target processing includes: performing a first processing on the input data using the neural network to obtain a first fixed-point number; determining a second fixed-point number corresponding to the first fixed-point number according to a fixed-point number correspondence; obtaining the target result based on the second fixed-point number; the fixed-point number correspondence includes a correspondence between the first fixed-point number and the second fixed-point number, the second fixed-point number being equal to a third fixed-point number obtained by performing a second processing on the first fixed-point number; the second processing includes: dequantizing the first fixed-point number to obtain a first floating-point number, processing the first floating-point number using a target nonlinear activation function to obtain a second floating-point number, and quantizing the second floating-point number to obtain the third fixed-point number; the target nonlinear activation function is the activation function used by the neural network; and an output module, configured to output the target result.

[0026] In one possible implementation, the processing module is specifically used to look up the second fixed-point number corresponding to the first fixed-point number in the fixed-point number correspondence table; the fixed-point number correspondence table includes the fixed-point number correspondence.

[0027] In one possible implementation, the first processing includes performing matrix multiplication on the input data using weight data. The first processing does not include quantization or dequantization operations. The input data includes fixed-point numbers, and the weight data includes fixed-point numbers.

[0028] In one possible implementation, the processing module is specifically used to perform a first normalization process on the second fixed-point number to obtain a fourth fixed-point number; the first normalization process includes: calculating the value of a first formula to obtain a third floating-point number, wherein the parameters in the first formula include a floating-point number obtained by data type conversion of the second fixed-point number; quantizing the third floating-point number to obtain a fifth fixed-point number; calculating the sum of the fifth fixed-point number and the sixth fixed-point number to obtain the fourth fixed-point number; the first formula is a part of a first normalization formula corresponding to the first normalization process, and the sixth fixed-point number is an offset value in the first normalization formula; and obtaining the target result based on the fourth fixed-point number.

[0029] In one possible implementation, the target nonlinear activation function is obtained by multiplying the scaling factor in the first normalization formula by the initial nonlinear activation function.

[0030] In one possible implementation, the processing module is specifically used to perform a second normalization process on the second fixed-point number to obtain a seventh fixed-point number; the second normalization process includes: calculating the value of a second normalization formula to obtain a fourth floating-point number, wherein the parameters in the second normalization formula include a floating-point number obtained by data type conversion of the second fixed-point number; quantizing the fourth floating-point number to obtain the seventh fixed-point number; and obtaining the target result based on the seventh fixed-point number.

[0031] In one possible implementation, the target nonlinear activation function is obtained by multiplying the scaling factor in the second normalization formula by the initial nonlinear activation function.

[0032] In one possible implementation, the neural network is any one of a recurrent neural network (RNN), a gated recurrent unit (GRU), a long short-term memory (LSTM), a bidirectional long short-term memory (BiLSTM), a simple recurrent unit (SRU), and future RNN variants.

[0033] In one possible implementation, the plurality of computer-processable signals include at least one of: voice signals, text signals, or image signals.

[0034] For the technical effects of the second aspect or various possible implementation methods, please refer to the introduction of the technical effects of the first aspect or corresponding implementation methods.

[0035] Thirdly, this application provides yet another data processing apparatus, which includes: a processor, a memory, and an output device. The memory is used to store code; the processor reads the code stored in the memory to execute the method provided in the first aspect or various possible implementations; and the output device is used to output the target result.

[0036] Fourthly, embodiments of this application provide a computer program product, the computer program product including program instructions, which, when executed by a processor, cause the processor to perform the first aspect or various possible implementations described above.

[0037] Fifth aspect: This application provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the methods provided in the first aspect or various possible embodiments described above. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application or the background art will be described below.

[0039] Figures 1A to 1C Application scenarios for natural language processing systems;

[0040] Figure 2 A comparative schematic diagram illustrating a normalization method provided in an embodiment of this application;

[0041] Figure 3 A schematic diagram illustrating an example of a data processing method provided in this application embodiment;

[0042] Figure 4 A flowchart of a data processing method provided in an embodiment of this application;

[0043] Figure 5 A schematic diagram illustrating an example of using a trained neural network to perform target processing on input data, provided in an embodiment of this application;

[0044] Figure 6 A schematic diagram illustrating part of the process of training a neural network using training samples, as provided in an embodiment of this application;

[0045] Figure 7 A flowchart of a natural language processing method provided in this application embodiment;

[0046] Figure 8 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0047] Figure 9 This is a schematic diagram of the structure of a neural network processor provided in an embodiment of this application;

[0048] Figure 10 A block diagram illustrating a partial structure of a terminal device provided in an embodiment of this application;

[0049] Figure 11 This is a schematic diagram of the structure of a server 1100 provided in an embodiment of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described below in conjunction with the accompanying drawings.

[0051] The terms "first" and "second," etc., used in the specification, claims, and drawings of this application are used only to distinguish different objects and not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0052] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0053] In this application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three or more, and "and / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items. For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c".

[0054] The data processing method provided in this application can be applied to scenarios such as image processing, speech recognition, and natural language processing, and is especially suitable for scenarios where neural networks (e.g., recurrent neural networks) are used to process time-related features.

[0055] The following describes the scenarios in which this application solution can be applied.

[0056] like Figure 1A As shown, a natural language processing system includes a user device and a data processing device.

[0057] The aforementioned user devices can be terminal devices such as mobile phones, personal computers, tablets, wearable devices, personal digital assistants, game consoles, and information processing centers. These user devices are the initiators of natural language processing tasks. As the initiator of natural language processing tasks (such as translation tasks, paraphrasing tasks, etc.), users typically initiate natural language processing tasks through these user devices. A paraphrasing task is the task of converting one natural language text into another text with the same meaning but a different expression. For example, "What makes the second world war happen" can be paraphrased as "What is the reason of world war II".

[0058] The aforementioned data processing device can be a cloud server, network server, application server, or management server, or any device or server with data processing capabilities. This data processing device receives query statements / voice / text questions from the aforementioned terminal devices via an interactive interface, and then performs language data processing through a storage device and a processor to perform machine learning, deep learning, search, reasoning, and decision-making. The aforementioned storage device can be a general term, including local storage and a database storing historical data; this database can be located on the data processing device or on other network servers.

[0059] like Figure 1B This illustrates another application scenario for a natural language processing system. In this scenario, the terminal device directly acts as a data processing unit, receiving input from the user and processing it directly through its own hardware. The specific process is similar to... Figure 1A Similar to the description above, it will not be repeated here.

[0060] like Figure 1C As shown, the user equipment can be local device 101 or 102, and the data processing device can be execution device 210. The data storage system 250 can be integrated on the execution device 210 or set up on the cloud or other network servers.

[0061] The data processing method of this application embodiment is applied to natural language processing scenarios, which can improve the speed of performing natural speech processing tasks and reduce storage space overhead. It should be understood that the data processing method of this application embodiment can also be applied to other scenarios that use neural networks to perform prediction tasks (such as image processing tasks), and can also improve the speed of performing prediction tasks.

[0062] Since the embodiments of this application involve a large number of neural network applications, for ease of understanding, the relevant terms and concepts such as neural networks involved in the embodiments of this application will be introduced below.

[0063] (1) Normalization processing

[0064] The essence of neural network learning is learning data distribution. Without normalization, the distribution of training data varies from batch to batch. From a macro perspective, the neural network needs to find an equilibrium point among these multiple distributions. From a micro perspective, because the distribution of input data for each layer is constantly changing, each layer is also searching for an equilibrium point, making convergence difficult. Of course, simply normalizing the input data (e.g., dividing the input image by 255 to normalize it to between 0 and 1) only guarantees a consistent distribution for the input layers, not for each layer. Therefore, normalization is also necessary in the intermediate layers of the neural network.

[0065] Commonly used normalization methods include batch normalization (BN), layer normalization (LN), instance normalization (IN), and group normalization (GN).

[0066] The calculation process for the four normalization methods, BN, LN, IN, and GN, is almost identical and can be divided into four steps:

[0067] 1) Calculate the mean of the given dataset using the following formula:

[0068]

[0069] Where μ represents the mean of a given dataset X = x1, x2, ..., x m , m represents the number of values ​​(corresponding to scalars) in the given dataset, x i This represents the i-th value in a given dataset X.

[0070] 2) Calculate the variance of the given dataset using the following formula:

[0071]

[0072] Where, σ 2 Let X represent the variance of a given dataset, and μ represent the mean of the given dataset. Given a dataset X = x1, x2, ..., x... m , m represents the number of values ​​(corresponding to scalars) in the given dataset, x i This represents the i-th value in a given dataset X.

[0073] 3) Normalize the values ​​in the given dataset to a mean of 0 and a variance of 1, using the following formula:

[0074]

[0075] in, Indicates normalization of x i The resulting value, ∈, is a tiny constant to prevent division by zero.

[0076] 4) Reconstruct the distribution learned by this layer of the network, as shown in the following formula:

[0077]

[0078] Among them, y i For normalization of x i The final values ​​obtained, γ and β, are those learned by the neural network during the training phase. After normalization, further adjustments are needed. Perform scaling and shifting. The parameters γ and β are used for learning.

[0079] Combining formulas (3) and (4), it can be seen that the four normalization processes BN, LN, IN, and GN satisfy the following formula:

[0080]

[0081] Among them, y i For normalization of x i The final value obtained. It should be understood that formula (5) can be written as x′ represents the normalized value of x. i The final value obtained.

[0082] Assume the feature map shape is denoted as [N, C, H, W], where N represents the batch size (N samples), C represents the number of channels, and H and W represent the height and width of the feature map, respectively. The main difference between these methods lies in:

[0083] 1. BN normalizes N, H, and W in a batch while preserving the dimension of channel C;

[0084] 2. LN normalizes C, H, and W along the channel direction, and its effect is particularly noticeable on RNNs;

[0085] 3. IN normalizes H and W at the image pixel level and is used for style transfer;

[0086] 4. GN groups the channels and then normalizes them.

[0087] Figure 2 This is a comparative schematic diagram illustrating a normalization method provided in an embodiment of this application. Figure 2 In the diagram, N represents the batch size (corresponding to the number of samples), C represents the channel, and (H, W) represents the height and width of the feature map. The feature map represented by 201 corresponds to the normalization method of BN, the feature map represented by 202 corresponds to the normalization method of LN, the feature map represented by 203 corresponds to the normalization method of IN, and the feature map represented by 204 corresponds to the normalization method of GN. Figure 2 In this process, the mean and variance are calculated using the values ​​of the darker parts, thus performing normalization.

[0088] from Figure 2 As can be seen, BN calculation involves normalizing the NHW (corresponding to a given dataset) of each channel separately. For each channel, there is a set of γ and β, so the learnable parameters are 2*C. LN calculation involves normalizing each CHW (corresponding to a given dataset) separately, unaffected by batch size. LN calculation can be used in RNN networks. IN calculation involves normalizing each HW (corresponding to a given dataset) separately, unaffected by channel and batch size. GN calculation involves first dividing channel C into G groups, then normalizing each (C / G) HW (corresponding to a given dataset) separately, and finally merging the normalized data of the G groups into CHW.

[0089] (2) Neural Network

[0090] A neural network can be composed of neural units, which can be operational units that take xs and an intercept of 1 as inputs, and whose output can be:

[0091]

[0092] Where s = 1, 2, ..., n, where n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of this activation function can be used as the input of the next convolutional layer. The activation function can be the sigmoid function, tanh function, etc. A neural network is a network formed by connecting many of the above-mentioned individual neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, which can be a region composed of several neural units.

[0093] (3) Recurrent Neural Networks (RNNs) are used to process sequential data. In traditional neural network models, the layers from the input layer to the hidden layer and then to the output layer are fully connected, but the nodes within each layer are unconnected. While this type of ordinary neural network has solved many difficult problems, it is still powerless against many others. For example, to predict the next word in a sentence, you generally need to use the preceding words because the words in a sentence are not independent. RNNs are called recurrent neural networks because the current output of an RNN is related to the previous output. Specifically, the recurrent neural network memorizes previous information and applies it to the calculation of the current output. That is, the nodes within the hidden layer are no longer unconnected but connected, and the input of the hidden layer includes not only the output of the input layer but also the output of the hidden layer at the previous time step. Theoretically, RNNs can process sequential data of any length. Training an RNN is the same as training a traditional CNN. The backpropagation algorithm is also used, but with one difference: if the RNN is expanded, the parameters, such as W, are shared; however, this is not the case for traditional neural networks as illustrated above. Furthermore, in the gradient descent algorithm, the output at each step depends not only on the network at the current step but also on the state of the network in the previous several steps. This learning algorithm is called Backpropagation Through Time (BPTT).

[0094] Since we already have convolutional neural networks (CNNs), why do we need recurrent neural networks (RNNs)? The reason is simple. CNNs rely on the fundamental assumption that elements are independent of each other, and that input and output are also independent—like a cat and a dog. However, in the real world, many elements are interconnected. For example, stock prices fluctuate over time. Or, imagine someone saying, "I love traveling, and my favorite place is Yunnan. I definitely want to go there someday." Humans know the answer to this question is "Yunnan." Humans can infer from context, but how can machines do the same? This is where RNNs come in. RNNs aim to give machines the ability to remember, just like humans. Therefore, the output of an RNN depends on both the current input information and historical memory information.

[0095] (4) Fixed-point arithmetic

[0096] In a given Cartesian coordinate system, points whose coordinates are all integers are called integer points; the group consisting of all integer points is called a spatial grid. Operations performed on a spatial grid are called fixed-point operations.

[0097] The preceding section introduced the relevant terms and concepts related to neural networks involved in the embodiments of this application. The following section, in conjunction with the accompanying drawings, describes a method for improving the inference speed of neural networks (i.e., improving the speed at which neural networks perform prediction tasks) by converting matrix multiplication performed by neural networks (e.g., RNNs) from floating-point operations to fixed-point operations.

[0098] Figure 3 This is a schematic diagram illustrating an example of a data processing method provided in an embodiment of this application. Figure 3 The process can be understood as the process by which a data processing device uses a neural network to perform a prediction task (i.e., the reasoning process). Figure 3 In this context, W1 represents weight data (e.g., a weight matrix), where W1 is a fixed-point number (e.g., int8, uint8, etc.); X1 represents input data (e.g., image data, speech data, etc.), where X1 is a floating-point number (e.g., float, etc.); quantization converts the input data X1 from a floating-point number (e.g., float, etc.) to a fixed-point number (e.g., uint8, etc.) X1'; matrix multiplication calculates the product of weight data W1 and X1' to obtain A1 (a fixed-point number); dequaztization converts A1 from a fixed-point number to a floating-point number A1'; addition calculates the sum of A1' and the offset value B1 (a floating-point number) to obtain C1 (a floating-point number); activation represents the use of non-linear activation functions such as tanh and sigmoid. The function C1 is processed to obtain D1 (a floating-point number); normalization means normalizing D1 to obtain E1 (a floating-point number). The normalization operation is optional, not mandatory. Figure 3 As can be seen, neural networks (such as RNNs) employ quantization techniques to quantize the input data online before performing matrix multiplication; then, the results of the matrix multiplication are dequantized back to floating-point numbers. By converting matrix multiplication into fixed-point operations, the inference speed of neural networks can be improved. However, Figure 3 The data processing methods in this application not only require frequent quantization and dequantization operations but also a large number of floating-point operations, resulting in high computational complexity. Since the computational complexity of floating-point operations is far greater than that of fixed-point operations, converting floating-point operations in the data processing methods to fixed-point operations, while maintaining computational accuracy, can effectively improve computational efficiency, i.e., increase inference speed. The fully fixed-point data processing method provided in this application embodiment can use fixed-point operations throughout the entire processing, which not only reduces computational complexity but also ensures computational accuracy.

[0099] Figure 4This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 4 As shown, the method includes:

[0100] 401. The data processing device uses a trained neural network to process the input data to obtain the target result.

[0101] The input data includes multiple computer-processable signals, including at least one of speech signals, text signals, or image signals. That is, the input data can be any of image data, speech data, text data, etc. The neural network can be a recurrent neural network (RNN), a gated recurrent unit (GRU), a long short-term memory (LSTM), a bidirectional long short-term memory (BiLSTM), or other neural networks suitable for processing time-related features; it can also be other neural networks employing non-linear activation functions, such as convolutional neural networks. The aforementioned target processing includes: performing a first processing on the input data using the aforementioned neural network to obtain a first fixed-point number; determining a second fixed-point number corresponding to the first fixed-point number based on the fixed-point number correspondence; and obtaining the aforementioned target result based on the second fixed-point number. The fixed-point number correspondence includes the correspondence between the first fixed-point number and the second fixed-point number, wherein the second fixed-point number is equal to a third fixed-point number obtained by performing a second processing on the first fixed-point number. The second processing includes: dequantizing the first fixed-point number to obtain a first floating-point number; processing the first floating-point number using a target nonlinear activation function to obtain a second floating-point number; and quantizing the second floating-point number to obtain the third fixed-point number. The target nonlinear activation function is the activation function used by the aforementioned neural network. In this application, a fixed-point number refers to an integer, such as an integer of type int8, uint8, int16, uint16, int32, uint32, etc. The first fixed-point number can be a single fixed-point number or all fixed-point numbers obtained by performing the first processing on the input data. In some embodiments, the aforementioned target processing further includes a normalization operation (e.g., Batch Normalization). Since Batch Normalization (BN) can be decomposed offline into multiply-accumulate and activation functions, the data processing device can normalize the second fixed-point number by decomposing it into BN and activation functions after executing the activation function. The data processing device can be a personal computer, computer workstation, smartphone, tablet, smart camera, smart car, or other type of cellular phone, media consumption device, wearable device, set-top box, game console, augmented reality (AR) device, virtual reality (VR) device, or other terminal device, and can also be a server. The input data is a fixed-point number, such as int8 type data. In some embodiments, the target nonlinear activation function is a nonlinear activation function with upper and lower bounds, such as the sigmoid function, tanh function, etc.

[0102] Optionally, before executing step 401, the data processing device quantizes the original input data to obtain fixed-point input data. Optionally, the input data acquired or received by the data processing device is fixed-point data. The first processing of the input data using the neural network may include: performing convolution operations (corresponding to matrix multiplication) and pooling operations on the input data using the neural network. Optionally, the first processing includes performing matrix multiplication operations on the input data using weight data. The first processing does not include quantization and dequantization operations. The values ​​included in the input data are all fixed-point numbers (i.e., integers), and the values ​​included in the weight data are all fixed-point numbers. In other words, the entire process of the data processing device performing the first processing of the input data using the neural network can employ fixed-point arithmetic.

[0103] One possible implementation of determining the second fixed-point number corresponding to the first fixed-point number based on the fixed-point number correspondence is as follows: The second fixed-point number corresponding to the first fixed-point number is searched in the fixed-point number correspondence table; the fixed-point number correspondence table includes the fixed-point number correspondence. The data processing device may pre-store the fixed-point number correspondence table or obtain it from other devices (e.g., a server). Optionally, each fixed-point number obtained by the data processing device through the first processing of any input data using the neural network is included in the fixed-point number correspondence table. That is, the data processing device can find the fixed-point number corresponding to any fixed-point number obtained by performing the first processing on the input data from the fixed-point number correspondence table. The second fixed-point number corresponding to the first fixed-point number in the fixed-point number correspondence table can be understood as a fixed-point number obtained by the following operations: dequantizing the first fixed-point number to obtain a first floating-point number, processing the first floating-point number using a target nonlinear activation function to obtain a second floating-point number, and quantizing the second floating-point number to obtain the second fixed-point number. It should be understood that the data processing device achieves the purpose of performing a second processing on the first fixed-point number by performing the operation of determining the second fixed-point number corresponding to the first fixed-point number according to the fixed-point number correspondence relationship. It does not need to perform the operation of "dequantizing the first fixed-point number to obtain a first floating-point number, processing the first floating-point number using a target nonlinear activation function to obtain a second floating-point number, and quantizing the second floating-point number to obtain the third fixed-point number".

[0104] 402. The data processing device outputs the above target results.

[0105] In some embodiments, the data processing device is a server, and one implementation of step 402 is as follows: the data processing device sends the target result (e.g., speech recognition result, paraphrase result, translation result, etc.) to the terminal device. In some embodiments, the data processing device is a terminal device, and one implementation of step 402 is as follows: the data processing device displays the target result (e.g., speech recognition result, paraphrase result, translation result, etc.) on a display screen or plays the target result (e.g., paraphrase result, translation result, etc.) through an audio device.

[0106] It should be understood that the data processing device replaces the following operations—dequantizing the first fixed-point number to obtain a first floating-point number, processing the first floating-point number using a target nonlinear activation function to obtain a second floating-point number, and quantizing the second floating-point number to obtain the second fixed-point number—by looking up the second fixed-point number corresponding to the first fixed-point number in the fixed-point number correspondence table. This significantly reduces the computational load, improves computational efficiency, and reduces the storage space occupied. Furthermore, the data processing device uses fixed-point arithmetic throughout step 401 and does not require quantization and dequantization operations, further improving computational efficiency.

[0107] In this embodiment, the second fixed-point number corresponding to the first fixed-point number is determined according to the fixed-point number correspondence; this can reduce computational complexity and improve inference speed.

[0108] The following is an example of using a trained neural network to process input data.

[0109] Figure 5 This is a schematic diagram illustrating an example of using a trained neural network to perform target processing on input data, as provided in an embodiment of this application. Figure 5 As shown, 501 represents matrix multiplication, which calculates A2 by multiplying the input data X2 and the weight data W2; 502 represents addition, which calculates the sum of A2 and the offset value B2 to obtain C2 (a fixed-point number); 503 represents a lookup operation, which finds D2 (a fixed-point number) corresponding to C2 in the fixed-point number correspondence table; normalization represents normalizing D2 to obtain E2. Normalization is optional, not mandatory. Here, X2, W2, B2, C2, and E2 are all fixed-point numbers, and... Figure 3 All data processing operations in the system employ fixed-point arithmetic, meaning the entire target processing process is fully fixed-point quantized.

[0110] In some embodiments, the first processing of the input data using the neural network described above to obtain the first fixed-point number can correspond to... Figure 5 501 and / or 502; based on the fixed-point number correspondence, determine the second fixed-point number corresponding to the first fixed-point number mentioned above. Figure 5503 in; based on the second fixed point number mentioned above, the target result corresponds to... Figure 5 504 errors and other operations. Figure 4 In the data processing method, determining the second fixed point corresponding to the first fixed point based on the fixed point correspondence can be achieved by looking up a table, for example, by looking up the second fixed point corresponding to the first fixed point in the fixed point correspondence table. Figure 5 The technical objective achieved by the lookup table operation 503 is equivalent to: dequantizing C2 to obtain C2' (floating-point number); processing C2' using non-linear activation functions such as tanh and sigmoid to obtain C2' (floating-point number); and quantizing C2' to obtain D2 (fixed-point number). It should be understood that in the data processing method provided in this application embodiment, replacing the dequantization operation on the input value of the activation function (corresponding to C2), the activation operation implemented using the activation function, and the quantization operation on the output value of the activation function (corresponding to C2') with a lookup table operation can effectively reduce the amount of computation and the consumption of storage resources. For example, from a technical objective perspective, the operation of determining the second fixed-point number corresponding to the first fixed-point number based on the fixed-point number correspondence is equivalent to the following formula:

[0111] X out =Q(Act(De(X) in ))) (7);

[0112] Among them, X in Let X represent the first fixed-point number. out The second fixed-point number is represented by De(), the dequantization formula (corresponding to the dequantization operation) is represented by Act(), the activation function (corresponding to the activation operation) is represented by Q(), and the quantization formula (corresponding to the quantization operation) is represented by Act(). Act() can be an activation function with upper and lower bounds, such as the sigmoid function or the tanh function.

[0113] The following describes some possible ways to obtain the correspondence between fixed-point numbers.

[0114] Method 1

[0115] The data processing device trains a neural network using training samples and records the quantization parameters performed during the processing of the training samples by the neural network, including quantization and / or dequantization operations. Based on the quantization parameters, the first fixed-point number (i.e., X) is calculated using formula (7). in ) and the second fixed-point number (i.e., X) out The correspondence between ) is the correspondence between fixed-point numbers. One possible quantization formula (i.e., Q()) is as follows:

[0116]

[0117] Where s is the quantization scaling factor, typically 2. k -1, where k is the quantization precision, and k is an integer greater than 1; 0 is the offset of the 0 bits before and after x quantization. The above quantization parameters can include one or more of max(x), min(x), and s, where max(x) represents the maximum value in the data processed by the quantization operation, and min(x) represents the minimum value in the data processed by the quantization operation.

[0118] Accordingly, the inverse quantization formula is as follows:

[0119] x=αx q +o (9);

[0120] in, O represents the offset of the 0 bits before and after x is quantized. The following is an example of a data processing device training a neural network using training samples.

[0121] Figure 6 This is a schematic diagram illustrating a portion of the process of training a neural network using training samples, as provided in an embodiment of this application. Figure 6 The process of training a neural network using training samples includes the following operations: calculating the sum of A3' (e.g., A3' is the result of matrix multiplication) and B3 (corresponding to the offset value) (corresponding to the Add operation) to obtain C3 (a floating-point number); quantizing C3 to obtain C3', and saving the first quantization parameter (i.e., the parameter used to quantize C3); dequantizing C3' using the first quantization parameter to obtain C3; processing C3 using an activation function to obtain D3 (a floating-point number); quantizing D3 to obtain D3', and saving the second quantization parameter (i.e., the parameter used to quantize D3); dequantizing D3' using the second quantization parameter to obtain D3. In some embodiments, the quantization parameter in the above formula (7) includes the above-mentioned first quantization parameter and the above-mentioned second quantization parameter. Figure 6 It can be seen that when the data processing device trains the neural network using training samples, it uses floating-point calculations throughout the process. After quantization and dequantization operations, it records the overall quantization parameters (i.e., the first and second quantization parameters) during training. It should be understood that... Figure 6 Merging the operations within the dashed box yields: X out =Q(Act(De(X) in ))), that is, formula (7); where C3' corresponds to X in D3' corresponds to X out Due to the input of the nonlinear activation function with upper and lower bounds (corresponding to X) in ) and output (corresponding to X) outThis can be equivalent to a one-to-one correspondence (i.e., a fixed-point number correspondence). From a technical perspective, Figure 5 The table lookup operation 503 in the code is equivalent to... Figure 6 The operations within the dashed box. For example, from a technical perspective, Figure 5 The table lookup operation 503 in the code is equivalent to X. out =Q(sigmoid(De(X) in ))) or X out =Q(tanh(De(X) in In other words, the data processing method provided in this application uses... Figure 5 The table lookup operation in the middle has replaced Figure 6 The dequantization, activation, and quantization operations are within the dashed box. Figure 6 The operations within the dashed box can be computed offline, and because the nonlinear activation function has upper and lower bounds, the input and output can be computed to obtain a fixed-point number (corresponding to X). in ) to a fixed point number (corresponding to X) out The one-to-one correspondence between the fixed-point numbers is the fixed-point number correspondence. The data processing device or other equipment can use the quantization parameters recorded during the training of the neural network and formula (7) to determine the fixed-point number correspondence. Optionally, the data processing device can use all the fixed-point numbers that may be obtained by performing the first processing on the input data using the above-mentioned neural network as input values ​​and substitute them into formula (7) to calculate, obtain the fixed-point number corresponding to each fixed-point number, and store the correspondence between the fixed-point numbers, that is, the fixed-point number correspondence.

[0122] Method 2

[0123] The data processing device trains a neural network using training samples and records the correspondence between the fixed-point numbers corresponding to the input values ​​of the activation function and the fixed-point numbers corresponding to the output values ​​of the activation function during the processing of the training samples by the neural network, thus obtaining the fixed-point number correspondence. Figure 6Taking the example of training a neural network, the data processing device can record the correspondence between C3' and D3'; where C3' (corresponding to the first fixed-point number) is the fixed-point number corresponding to the input value C3 of the activation function, that is, C3 can be obtained by dequantizing C3'; and D3' (corresponding to the second fixed-point number) is the fixed-point number corresponding to the output value D3 of the activation function, that is, D3 can be obtained by quantizing D3. In some embodiments, the data processing device can use a large number of training samples to train the neural network and record the correspondence between the fixed-point number corresponding to the input value of the activation function and the fixed-point number corresponding to the output value of the activation function during the processing of the training samples by the neural network, thus obtaining the fixed-point number correspondence. For example, when training a neural network using the first training sample, the data processing device records the correspondence between the fixed-point numbers corresponding to the input values ​​of 100 activation functions and the fixed-point numbers corresponding to the output values ​​of those activation functions. When training the neural network using the second training sample, it records the correspondence between the fixed-point numbers corresponding to the input values ​​of activation functions that were not previously recorded and the fixed-point numbers corresponding to the output values ​​of those activation functions (i.e., new correspondences). This training is repeated multiple times until no new correspondence between the fixed-point numbers corresponding to the input values ​​of activation functions and the fixed-point numbers corresponding to the output values ​​of those activation functions can be obtained. It should be understood that when training a neural network using a large number of training samples, the data processing device can record the correspondence between the fixed-point numbers corresponding to any input value of an activation function and the fixed-point numbers corresponding to the output values ​​of that activation function, i.e., the fixed-point number correspondence.

[0124] By training with pseudo-fixed-point quantization (e.g.) Figure 6 During the training process, the dequantization and quantization operations before and after the activation function are transformed into fixed-point lookup tables after mathematical equivalence transformation in offline mode, which can solve the fixed-point calculation problem of nonlinear activation functions.

[0125] The foregoing embodiments describe a scheme that employs fixed-point computation throughout the prediction process using neural networks, i.e., a full-process fixed-point scheme. In some embodiments, the data processing device needs to perform normalization operations, such as BN, LN, IN, GN, etc., during the prediction process using neural networks. For example, when a data processing device uses RNNs and their variants to perform a prediction task, it typically needs to perform a normalization operation, such as LN and BN, to improve the stability of the neural network model. Since normalization operations such as LN need to be computed online and cannot be decomposed offline into multiply-accumulate and activation functions like BN, this interrupts the fixed-point data flow and increases quantization and dequantization operations. Below are some possible implementation methods for obtaining the aforementioned target result based on the second fixed-point number.

[0126] A general normalization formula is as follows:

[0127]

[0128] Where μ represents the mean of the given dataset, σ 2 Let γ represent the variance of the given dataset, β represent the scaling factor (corresponding to scaling), β represent the offset (corresponding to offset), x represent any number in the given dataset, x′ represent the value obtained by normalizing x, γ and β are learned by the neural network during the training phase, and ∈ is a fixed value. Batch Normalization (BN) calculates by normalizing the NHW (corresponding to the given dataset) of each channel individually. For each channel, there is a set of γ and β, so the learnable parameters are 2*C. Loop Normalization (LN) calculates by normalizing each CHW (corresponding to the given dataset) individually, unaffected by batch size. LN calculations can be used in RNN networks. Induction Normalization (IN) calculates by normalizing each HW (corresponding to the given dataset) individually, unaffected by channel and batch size. Ginduction Normalization (GN) calculates by first dividing channel C into G groups, then normalizing each (C / G) HW (corresponding to the given dataset) individually, and finally merging the normalized data from the G groups into CHW. In some embodiments, the data processing device may calculate the mean and variance based on each batch of data (i.e., a given dataset) in the training samples when training the neural network; in the application phase (i.e., the phase of using the neural network to perform a prediction task), for the mean, the data processing device may directly calculate the average of all the means obtained during training (e.g., the batch mean), and for the standard deviation (i.e., ... The data processing device can use an unbiased estimate of the variances (e.g., batch variances) obtained during training.

[0129] In formula (10), γ can be incorporated into the activation function. For example, the initial nonlinear activation function (i.e., the original activation function of the neural network) can be multiplied by γ to obtain the target nonlinear activation function. The function of the activation function is implemented using the lookup table method in the aforementioned embodiment. β can be directly represented using fixed-point numbers. Therefore, only the following needs to be considered: Point-to-point processing enables point-to-point processing throughout the entire data processing workflow.

[0130] One possible quantization formula is as follows:

[0131]

[0132] Where s is the quantization scaling factor, typically 2. k -1, where k is the quantization precision, and k is an integer greater than 1; 0 is the offset of the 0 bits before and after x quantization. Correspondingly, the dequantization formula is: x = αx q +o, where, Let x = αx q +o substitution get in, Since γ in formula (10) can be incorporated into the initial nonlinear activation function of the neural network to obtain the target nonlinear function, and β can be directly taken as a fixed-point number, the data processing device only needs to perform normalization processing according to formula (10). Fixed-point processing can make the entire process fixed-point. This is because x = ax q +o substitution get Therefore, the data processing device needs to calculate the full-process fixed-point positioning to achieve this. The corresponding quantized value (i.e., for) (Fixed-point numbers obtained through quantization).

[0133] When implementing a neural network using computer programming (corresponding to program code), operations such as exponentiation and open operations do not support fixed-point computation. Therefore, during normalization, fixed-point numbers can be directly converted to floating-point numbers (note that this conversion is not inverse quantization, but a direct data type conversion). Finally, after completing the normalization calculation, the output is requantized according to the maximum and minimum values ​​of the result (corresponding to the third floating-point number) using the general quantization formula (11) (corresponding to the fifth fixed-point number). In some embodiments, when a data processing device or other device trains a neural network, fixed-point operations can be used to calculate the mean μ and / or variance σ of a given dataset. 2 This can improve computational performance. In other words, the mean u of the numerator part... q Fixed-point numerical computation (corresponding to fixed-point arithmetic) can be used, and the difference between the results of fixed-point computation and the structure of floating-point computation is acceptable. The main benefit here is that integer-based mean calculation offers a considerable performance advantage over floating-point mean calculation. Similarly, using integer-based mean in variance calculation also improves performance.

[0134] Method 1

[0135] An optional implementation of obtaining the target result based on the second fixed-point number is as follows: The second fixed-point number is subjected to a first normalization process to obtain a fourth fixed-point number; the first normalization process includes: calculating the value of a first formula to obtain a third floating-point number, wherein the parameters in the first formula include a floating-point number obtained by data type conversion of the second fixed-point number; quantizing the third floating-point number to obtain a fifth fixed-point number; calculating the sum of the fifth and sixth fixed-point numbers to obtain the fourth fixed-point number; the first formula is a part of the first normalization formula corresponding to the first normalization process, and the sixth fixed-point number is an offset value in the first normalization formula; the target result is obtained based on the fourth fixed-point number.

[0136] Optionally, the first normalization formula above can satisfy the following formula:

[0137]

[0138] In formula (12), the meanings of the parameters are the same as those in formula (11), x″ represents the fourth fixed-point number mentioned above, γ represents the scaling factor, β represents the sixth fixed-point number mentioned above (corresponding to the offset value), and x q This represents the second fixed-point number mentioned above. This represents the third floating-point number mentioned above. Represents the fifth fixed-point number, float(x) q -u q ) indicates that (x q -u q Convert σ to a floating-point number, float(σ) q ) 2 This indicates that (σ) q ) 2 Convert to floating-point numbers, where c is a constant (can be calculated offline). The above calculation of the first formula to obtain the third floating-point number can be done as follows: Calculate... To obtain the third floating-point number. Optionally, the aforementioned target nonlinear activation function is obtained by multiplying the scaling factor α in the first normalization formula by the initial nonlinear activation function. That is, the scaling factor γ is incorporated into the nonlinear activation function, and the data processing device actually calculates... (i.e., the third floating-point number), then quantized. This will yield the fifth floating-point number. Optionally, the calculation of the first formula above to obtain the third floating-point number can be: Calculate... The third floating-point number is obtained, and γ is not incorporated into the activation function. It should be understood that the first normalization process described above is to normalize the second fixed-point number according to the above formula (12). In some embodiments, the data processing device may pre-calculate the quantization parameters (which may include max(x), min(x), s) recorded during the training of the neural network and the parameters required for normalization (e.g., μ, σ). 2 The parameters in formula (12) are obtained by calculating σ, etc., for example γ is then incorporated into the activation function to obtain the target nonlinear activation function, and β is quantized to a fixed-point number, where, In this way, the data processing device can obtain all the parameters needed to perform the first normalization process on the aforementioned second fixed-point number to obtain the fourth fixed-point number. For example, the data processing device converts x... q μ q c and σ q All fixed-point numbers in the table are converted to floating-point numbers (e.g., float(x)). q -u q ) and float(σq ) 2 ), and calculate The third floating-point number is obtained; the third floating-point number is quantized to obtain the fifth fixed-point number; the sum of the fifth and sixth fixed-point numbers (corresponding to β) is calculated to obtain the fourth fixed-point number (corresponding to x″). The fourth fixed-point number can be understood as the fixed-point number obtained by performing the first normalization process on the second fixed-point number. That is, in the process of performing the first normalization process on the second fixed-point number to obtain the fixed-point number using method one, only one necessary quantization operation is performed, and no dequantization operation is required.

[0139] In this implementation, the second fixed-point number is normalized by a first normalization formula obtained through mathematical transformation and merging of the normalization formula and the inverse quantization formula, retaining only one necessary quantization operation, which can reduce the computational complexity of the normalization operation and improve the computational efficiency.

[0140] Method 2

[0141] The above-mentioned target result is obtained based on the second fixed-point number as follows: the second fixed-point number is subjected to a second normalization process to obtain a seventh fixed-point number; the second normalization process includes: calculating the value of the second normalization formula to obtain a fourth floating-point number, wherein the parameters in the second normalization formula include a floating-point number obtained by data type conversion of the second fixed-point number; quantizing the fourth floating-point number to obtain the seventh fixed-point number; and obtaining the target result based on the seventh fixed-point number.

[0142] Optionally, the second normalization formula above can satisfy the following formula:

[0143]

[0144] In formula (13), x″′ represents the fourth fixed-point number mentioned above, and x in formula (13) q The second fixed-point number is represented by the parameter in formula (13) other than x″′, which has the same meaning as the parameter in formula (12). Calculating the value of the second normalization formula to obtain the fourth floating-point number can be done by calculating the value of formula (13) to obtain the fourth floating-point number. In some embodiments, γ and β in formula (13) are both floating-point numbers. In some embodiments, β in formula (13) is a fixed-point number and γ is a floating-point number. In some embodiments, β in formula (13) is a fixed-point number and γ is incorporated into the initial nonlinear activation function, where γ is a floating-point number.

[0145] In this implementation, the second fixed-point number is normalized by a second normalization formula obtained by mathematical transformation and merging of the normalization formula and the inverse quantization formula. Only one necessary quantization operation is retained, which can reduce the computational complexity of the normalization operation and improve the computational efficiency.

[0146] The proposed solution can be applied to a variety of scenarios. The following section describes how to perform natural language processing tasks using a data processing device. Figure 7 A flowchart of a natural language processing method provided in this application embodiment is shown below. Figure 7 As shown, the method may include:

[0147] 701. The data processing device obtains the natural language text to be processed.

[0148] The natural language text to be processed can be a sentence that the data processing device is currently processing. The data processing device can process the received natural language text or the natural language text obtained from speech recognition sentence by sentence.

[0149] exist Figure 1A and Figure 1C In this scenario, obtaining the natural language text to be processed can be achieved by a data processing device receiving voice or text data sent by a user device and obtaining the natural language text to be processed based on the received voice or text data. For example, the data processing device receives two sentences sent by the user device. The data processing device obtains the first sentence (the natural language text to be processed), processes the first sentence using a trained neural network, and outputs the result of processing the first sentence; it then obtains the second sentence (the natural language text to be processed), processes the second sentence using a trained deep neural network, and outputs the result of processing the second sentence.

[0150] exist Figure 1B In this scenario, obtaining the natural language text to be processed can be achieved by the terminal device directly receiving user input data such as voice or text, and then obtaining the natural language text based on the received voice or text data. For example, if the terminal device receives two sentences input by the user, it acquires the first sentence (the natural language text to be processed), processes it using a trained deep neural network, and outputs the result of processing the first sentence; it then acquires the second sentence (the natural language text to be processed), processes it using the trained deep neural network, and outputs the result of processing the second sentence.

[0151] 702. Use the trained recurrent neural network to perform target processing on the natural language text and output the target result obtained from processing the natural language text.

[0152] Recurrent neural networks can be replaced by any of GRU, LSTM, BiLSTM, etc. The target processing can be translation, paraphrasing, summarization, etc. The target result is another natural language text obtained by processing the natural language text. For example, the target result is a natural language text obtained by translating the natural language text. Another example is a natural language text obtained by paraphrasing the natural language text. The natural language text to be processed can be considered as an input sequence, and the target result (another natural language text) obtained by the data processing device can be considered as a generation sequence. The natural language text to be processed is... Figure 4 An example of the input data in the example. Figure 7 The target processing in is Figure 4 The target processing in the process. It should be understood that, Figure 7 The method flow is as follows Figure 4 An example of the method flow in [the context].

[0153] In this embodiment, the second fixed-point number corresponding to the first fixed-point number is determined according to the fixed-point number correspondence; this can reduce computational complexity and improve inference speed.

[0154] The foregoing embodiments described data processing methods; the following describes the structure of the data processing apparatus that implements these methods. Figure 8 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application, as shown below. Figure 8 As shown, the data processing apparatus may include:

[0155] Processing module 801 is used to perform target processing on input data using a trained neural network to obtain a target result. The input data includes multiple computer-processable signals. The target processing includes: performing a first processing on the input data using the neural network to obtain a first fixed-point number; determining a second fixed-point number corresponding to the first fixed-point number based on a fixed-point number correspondence; and obtaining the target result based on the second fixed-point number. The fixed-point number correspondence includes the correspondence between the first fixed-point number and the second fixed-point number, where the second fixed-point number is equal to a third fixed-point number obtained by performing a second processing on the first fixed-point number. The second processing includes: dequantizing the first fixed-point number to obtain a first floating-point number; processing the first floating-point number using a target nonlinear activation function to obtain a second floating-point number; and quantizing the second floating-point number to obtain the third fixed-point number. The target nonlinear activation function is the activation function used by the neural network.

[0156] Output module 802 is used to output the above target results.

[0157] In one optional implementation, the processing module 801 is specifically used to look up the second fixed-point number corresponding to the first fixed-point number in the fixed-point number correspondence table; the fixed-point number correspondence table includes the fixed-point number correspondence.

[0158] In one optional implementation, the first processing includes performing matrix multiplication on the input data using weight data. The first processing does not include quantization or dequantization operations. The input data includes fixed-point numbers, and the weight data includes fixed-point numbers.

[0159] In an optional implementation, processing module 801 is specifically used to perform a first normalization process on the second fixed-point number to obtain a fourth fixed-point number; the first normalization process includes: calculating the value of a first formula to obtain a third floating-point number, wherein the parameters in the first formula include a floating-point number obtained by data type conversion of the second fixed-point number; quantizing the third floating-point number to obtain a fifth fixed-point number; calculating the sum of the fifth and sixth fixed-point numbers to obtain the fourth fixed-point number; wherein the first formula is a part of a first normalization formula corresponding to the first normalization process, and the sixth fixed-point number is an offset value in the first normalization formula; and obtaining the target result based on the fourth fixed-point number.

[0160] In one alternative implementation, the target nonlinear activation function is obtained by multiplying the scaling factor in the first normalization formula with the initial nonlinear activation function.

[0161] In one optional implementation, the processing module 801 is specifically used to perform a second normalization process on the second fixed-point number to obtain a seventh fixed-point number; the second normalization process includes: calculating the value of a second normalization formula to obtain a fourth floating-point number, wherein the parameters in the second normalization formula include a floating-point number obtained by data type conversion of the second fixed-point number; quantizing the fourth floating-point number to obtain the seventh fixed-point number; and obtaining the target result based on the seventh fixed-point number.

[0162] In one alternative implementation, the target nonlinear activation function is obtained by multiplying the scaling factor in the second normalization formula above with the initial nonlinear activation function.

[0163] In one alternative implementation, the neural network described above is any one of a recurrent neural network (RNN), a gated recurrent unit (GRU), a long short-term memory (LSTM), or a bidirectional long short-term memory (BiLSTM).

[0164] In one alternative implementation, the aforementioned computer-processable signals include at least one of voice signals, text signals, or image signals.

[0165] The methods executed by the data processing device using a neural network in the foregoing embodiments can be implemented in a neural network processing unit (NPU). Figure 9 This is a schematic diagram of the structure of a neural network processor provided in an embodiment of this application.

[0166] The Neural Processing Unit (NPU) 90 is mounted as a coprocessor on the host CPU, which assigns tasks (such as natural language processing tasks) to it. The core of the NPU is the arithmetic circuit 90, which, through the controller 904, controls the arithmetic circuit 903 to retrieve matrix data from memory and perform multiplication operations.

[0167] In some implementations, the arithmetic circuit 903 internally includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 903 is a two-dimensional pulsating array. The arithmetic circuit 903 can also be a one-dimensional pulsating array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 903 is a general-purpose matrix processor.

[0168] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 902 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 901 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 908.

[0169] Unified memory 906 is used to store input and output data. Weight data is directly transferred to weight memory 902 via Direct Memory Access Controller (DMAC) 905. Input data is also transferred to unified memory 906 via DMAC.

[0170] The Bus Interface Unit (BIU) 510 is used for interaction between the AXI bus and the DMAC and the Instruction Fetch Buffer 909.

[0171] The bus interface unit 510 is also used by the instruction fetch memory 909 to fetch instructions from external memory, and by the memory access controller 905 to fetch the original data of the input matrix A or the weight matrix B from external memory.

[0172] The DMAC is mainly used to move input data from external memory DDR to unified memory 906, or to weight data to weight memory 902, or to input data to input memory 901.

[0173] The vector computation unit has 907 processing units that further process the output of the computation circuit as needed, such as vector multiplication, vector addition, exponential operations, logarithmic operations, size comparisons, etc. It is mainly used for computation in non-convolutional / FC layers of neural networks, such as pooling, batch normalization, and local response normalization.

[0174] In some implementations, the vector computation unit 907 can store the processed output vector into a unified buffer 906. For example, the vector computation unit 907 can apply a nonlinear function to the output of the arithmetic circuit 903, such as a vector of accumulated values, to generate activation values. In some implementations, the vector computation unit 907 generates normalized values, merged values, or both. In some implementations, the processed output vector can be used as activation input to the arithmetic circuit 903, for example, for use in subsequent layers of a neural network.

[0175] The instruction fetch buffer 909 connected to the controller 904 is used to store the instructions used by the controller 904;

[0176] The unified memory 906, input memory 901, weighted memory 902, and instruction fetch memory 909 are all on-chip memories.

[0177] in, Figure 3 The operations of each layer in the deep neural network shown can be performed by the matrix calculation unit 212 or the vector calculation unit 907. It should be understood that the data processing method performed by the neural network in the data processing device in the foregoing embodiment can also be implemented in other processors.

[0178] This application uses an NPU to implement a neural network-based data processing method, which can greatly improve the efficiency of data processing devices in performing prediction tasks using neural networks.

[0179] The data processing apparatus in the embodiments of the present invention will now be described from the perspective of hardware processing.

[0180] Figure 10 This is a block diagram illustrating a partial structure of a terminal device provided in an embodiment of this application. (Reference) Figure 10The terminal device includes components such as: radio frequency (RF) circuit 1010, memory 1020, input unit 1030, display unit 1040, sensor 1050, audio circuit 1060, wireless fidelity (WiFi) module 1070, system on chip (SoC) 1080, and power supply 1090. Figure 10 The terminal device can be an example of the data processing device in the foregoing embodiments.

[0181] The memory 1020 includes DDR memory, and may also include high-speed random access memory, or other storage units such as non-volatile memory, for example, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0182] Those skilled in the art will understand that Figure 10 The terminal device structure shown does not constitute a limitation on the terminal device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0183] The following is combined with Figure 10 A detailed introduction to each component of the terminal device:

[0184] The RF circuit 1010 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it for the SoC 1080; additionally, it transmits uplink data to the base station. Typically, the RF circuit 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the RF circuit 1010 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0185] The memory 1020 can be used to store software programs and modules. The SoC 1080 executes various functional applications and data processing of the terminal device by running the software programs and modules stored in the memory 1020. The memory 1020 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, translation function, repetition function, etc.), etc.; the data storage area may store data created according to the use of the terminal device (such as audio data, phone book, etc.).

[0186] The input unit 1030 can be used to receive input data (such as natural language text, voice data, etc.) and generate key signal inputs related to user settings and function control of the terminal device. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1031) and drive corresponding connection devices according to a pre-set program. For example, the touch panel 1031 is used to receive natural language text input by the user and input the natural language text to the SoC 1080. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, transmitting the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the SoC 1080. It can also receive and execute commands from the SoC 1080. Furthermore, the touch panel 1031 can be implemented using various types of sensors, including resistive, capacitive, infrared, and surface acoustic wave sensors. In addition to the touch panel 1031, the input unit 1030 may also include other input devices 1032. Specifically, other input devices 1032 may include, but are not limited to, one or more of the following: a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, a joystick, a touchscreen, and a microphone. The microphone included in the input device 1032 can receive user-inputted voice data and input the voice data to the SoC 1080.

[0187] The SoC 1080 executes the data processing method provided in this application to perform target processing on the input data input to the input unit 1030 by running the software program and modules stored in the memory 1020, thereby obtaining the target result. For example, after converting the speech data input to the input unit 1030 into natural language text, the SoC 1080 can execute the data processing method provided in this application to process the natural language text and obtain the target result.

[0188] Display unit 1040 can be used to display information input by the user or information provided to the user, as well as various menus of the terminal device. Display unit 1040 may include display panel 1041, optionally configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel. Display unit 1040 can be used to display the target results obtained by SoC 1080 processing input data. Further, touch panel 1031 may cover display panel 1041. When touch panel 1031 detects a touch operation on or near it, it transmits the information to SoC 1080 to determine the type of touch event. Subsequently, SoC 1080 provides corresponding visual output on display panel 1041 based on the type of touch event. Although in Figure 10 In this embodiment, the touch panel 1031 and the display panel 1041 are two separate components to realize the input and output functions of the terminal device. However, in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the terminal device.

[0189] The terminal device may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1041 according to the ambient light level, and the proximity sensor can turn off the display panel 1041 and / or backlight when the terminal device is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that identify the terminal device's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometers, taps), etc. Other sensors that may be configured in the terminal device, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0190] The audio circuit 1060, speaker 1061, and microphone 1062 provide an audio interface between the user and the terminal device. The audio circuit 1060 converts received audio data into electrical signals and transmits them to the speaker 1061, where the speaker 1061 converts them into sound signals for output. On the other hand, the microphone 1062 converts collected sound signals into electrical signals, which are then received by the audio circuit 1060, converted into audio data, and output to the SoC 1080 for processing. The audio data is then transmitted via the RF circuit 1010 to, for example, another terminal device, or output to the memory 1020 for further processing.

[0191] WiFi is a short-range wireless transmission technology. Terminal devices using the WiFi module 1070 can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 10 The WiFi module 1070 is shown, but it is understood that it is not a necessary component of the terminal device and can be omitted as needed without changing the essence of the invention.

[0192] The SoC 1080 serves as the control center of the terminal device, connecting various parts of the device via interfaces and lines. It executes software programs and / or modules stored in the memory 1020 and accesses data stored in the memory 1020 to perform various functions and process data, thereby providing overall monitoring of the terminal device. Optionally, the SoC 1080 may include multiple processing units, such as a CPU or various service processors (e.g., an NPU). The SoC 1080 may also integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the SoC 1080.

[0193] The terminal device also includes a power supply 1090 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the SoC 1080 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0194] Although not shown, the terminal device may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0195] Figure 11This is a schematic diagram of the structure of a server 1100 provided in an embodiment of this application. The server 1100 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1122 (e.g., one or more processors) and a memory 1132, and one or more storage media 1130 (e.g., one or more mass storage devices) for storing application programs 1142 or data 1144. The memory 1132 and storage media 1130 can be temporary or persistent storage. The program stored in the storage media 1130 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 1122 may be configured to communicate with the storage media 1130 and execute the series of instruction operations in the storage media 1130 on the server 1100. The server 1100 can be the data processing apparatus provided in this application.

[0196] Server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0197] The steps performed by the data processing device in the above embodiments can be based on this Figure 11 The server architecture is shown. Specifically, the central processing unit 1122 can implement... Figure 8 The processing module 801 performs the functions of the output module 802. The wireless network interface 1150 can also receive input data from terminal devices.

[0198] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, or they may be electrical, mechanical, or other forms of connection.

[0199] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the technical effects of the solutions provided in the embodiments of this application, depending on actual needs. Furthermore, the functional units in the various embodiments of this application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0200] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0201] In addition, this application also provides a computer program for implementing the operations and / or processes performed by the data processing device in the method embodiments provided in this application.

[0202] This application also provides a computer-readable storage medium storing computer code that, when executed on a computer, causes the computer to perform operations and / or processes performed by a data processing device in the method embodiments provided in this application.

[0203] This application also provides a computer program product, which includes computer code or a computer program that, when run on a computer, causes the operations and / or processes performed by a data processing device in the method embodiments provided in this application to be implemented.

[0204] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims. This application provides a computer-readable storage medium storing a computer program, which includes software program instructions. When these program instructions are executed by a processor in a data processing device, they implement the data processing method described in the foregoing embodiments.

[0205] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0206] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, include: The trained neural network is used to process the input data to obtain the target result. The input data includes multiple computer-processable signals. The target processing includes: performing a first processing on the input data using the neural network to obtain a first fixed-point number; determining a second fixed-point number corresponding to the first fixed-point number based on a fixed-point number correspondence; and obtaining the target result based on the second fixed-point number. The fixed-point number correspondence includes the correspondence between the first fixed-point number and the second fixed-point number, where the second fixed-point number is equal to a third fixed-point number obtained by performing a second processing on the first fixed-point number. The second processing includes: dequantizing the first fixed-point number to obtain a first floating-point number; processing the first floating-point number using a target nonlinear activation function to obtain a second floating-point number; and quantizing the second floating-point number to obtain the third fixed-point number. The target nonlinear activation function is the activation function used by the neural network. The multiple computer-processable signals include at least one of speech signals, text signals, or image signals. Output the target result.

2. The method according to claim 1, characterized in that, The step of determining the second fixed-point number corresponding to the first fixed-point number based on the fixed-point number correspondence includes: Find the second fixed-point number corresponding to the first fixed-point number in the fixed-point number correspondence table; the fixed-point number correspondence table includes the fixed-point number correspondences, and any correspondence in the fixed-point number correspondence table is the correspondence between the fixed-point number corresponding to the input value of the target nonlinear activation function and the fixed-point number corresponding to the output value of the target nonlinear activation function.

3. The method according to claim 1 or 2, characterized in that, The process of obtaining the target result based on the second fixed-point number includes: The second fixed-point number is subjected to a first normalization process to obtain a fourth fixed-point number; the first normalization process includes: calculating the value of a first formula to obtain a third floating-point number, wherein the parameters in the first formula include a floating-point number obtained by data type conversion of the second fixed-point number; quantizing the third floating-point number to obtain a fifth fixed-point number; calculating the sum of the fifth fixed-point number and the sixth fixed-point number to obtain the fourth fixed-point number; the first formula is a part of the first normalization formula corresponding to the first normalization process, and the sixth fixed-point number is an offset value in the first normalization formula; The target result is obtained based on the fourth fixed point number.

4. The method according to claim 3, characterized in that, The target nonlinear activation function is obtained by multiplying the scaling factor in the first normalization formula with the initial nonlinear activation function.

5. The method according to claim 1 or 2, characterized in that, The process of obtaining the target result based on the second fixed-point number includes: The second fixed-point number is subjected to a second normalization process to obtain a seventh fixed-point number; the second normalization process includes: calculating the value of a second normalization formula to obtain a fourth floating-point number, wherein the parameters in the second normalization formula include a floating-point number obtained by data type conversion of the second fixed-point number; and quantizing the fourth floating-point number to obtain the seventh fixed-point number. The target result is obtained based on the seventh fixed point.

6. The method according to claim 5, characterized in that, The target nonlinear activation function is obtained by multiplying the scaling factor in the second normalization formula by the initial nonlinear activation function.

7. A data processing apparatus, characterized in that, include: The processing module is used to process the input data using the trained neural network to obtain the target result. The input data includes multiple computer-processable signals. The target processing includes: performing a first processing on the input data using the neural network to obtain a first fixed-point number; determining a second fixed-point number corresponding to the first fixed-point number based on a fixed-point number correspondence; and obtaining the target result based on the second fixed-point number. The fixed-point number correspondence includes the correspondence between the first fixed-point number and the second fixed-point number, where the second fixed-point number is equal to a third fixed-point number obtained by performing a second processing on the first fixed-point number. The second processing includes: dequantizing the first fixed-point number to obtain a first floating-point number; processing the first floating-point number using a target nonlinear activation function to obtain a second floating-point number; and quantizing the second floating-point number to obtain the third fixed-point number. The target nonlinear activation function is the activation function used by the neural network. The multiple computer-processable signals include at least one of speech signals, text signals, or image signals. The output module is used to output the target result.

8. The data processing apparatus according to claim 7, characterized in that, The processing module is specifically used to look up the second fixed-point number corresponding to the first fixed-point number in the fixed-point number correspondence table; the fixed-point number correspondence table includes the fixed-point number correspondences, and any correspondence in the fixed-point number correspondence table is the correspondence between the fixed-point number corresponding to the input value of the target nonlinear activation function and the fixed-point number corresponding to the output value of the target nonlinear activation function.

9. The data processing apparatus according to claim 7 or 8, characterized in that, The processing module is specifically used to perform a first normalization process on the second fixed-point number to obtain a fourth fixed-point number; the first normalization process includes: calculating the value of a first formula to obtain a third floating-point number, wherein the parameters in the first formula include a floating-point number obtained by converting the data type of the second fixed-point number; quantizing the third floating-point number to obtain a fifth fixed-point number; calculating the sum of the fifth fixed-point number and the sixth fixed-point number to obtain the fourth fixed-point number; the first formula is a part of the first normalization formula corresponding to the first normalization process, and the sixth fixed-point number is an offset value in the first normalization formula; The target result is obtained based on the fourth fixed point number.

10. The data processing apparatus according to claim 9, characterized in that, The target nonlinear activation function is obtained by multiplying the scaling factor in the first normalization formula by the initial nonlinear activation function.

11. The data processing apparatus according to claim 7 or 8, characterized in that, The processing module is specifically used to perform a second normalization process on the second fixed-point number to obtain a seventh fixed-point number; The second normalization process includes: calculating the value of a second normalization formula to obtain a fourth floating-point number, wherein the parameters in the second normalization formula include a floating-point number obtained by data type conversion of the second fixed-point number; and quantizing the fourth floating-point number to obtain the seventh fixed-point number. The target result is obtained based on the seventh fixed point.

12. The data processing apparatus according to claim 11, characterized in that, The target nonlinear activation function is obtained by multiplying the scaling factor in the second normalization formula by the initial nonlinear activation function.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 6.

Images