Quantization method and device of neural network model, and data processing method and device

Abstract

A neural network model quantization method and device, and a data processing method and device, belong to the field of artificial intelligence. The original neural network model includes a first operator, a second operator and a first operation module, the first operation module is used for operating the output of the first operator and the output of the second operator, the neural network model quantization method comprises: determining a data quantization parameter according to the range of the first training input data of the first operator and the range of the second training input data of the second operator; determining a quantized neural network model, the quantized neural network model uses the data quantization parameter to quantize the first input data of the quantized first operator and the second input data of the quantized second operator respectively. The processing result of the quantized first operator and the processing result of the quantized second operator can be directly operated, which improves the data processing accuracy of the neural network model and improves the data processing efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and more particularly, to a neural network model quantization method and device, and a data processing method and device. BACKGROUND

[0002] Artificial intelligence (AI) is the use of digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that aims to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is the design principle and implementation method of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making. The research in the field of artificial intelligence includes robots, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, AI basic theory, etc.

[0003] Neural network models are widely used. By quantizing the operators in the neural network model, i.e., quantizing the parameters of the operators and quantizing the input data, the floating-point operation can be converted into the operation on fixed-point numbers, and the model size, inference speed and power consumption, etc. can be obtained. According to the data range of the operator, the quantization parameter of the operator is determined, which can improve the accuracy of the data processing result of the quantized operator. However, before the subsequent operation processing of the data processing result of the quantized multiple operators, the data processing result needs to be dequantized, which leads to poor overall processing performance. SUMMARY

[0004] The present application provides a neural network model quantization method and a data processing method, which can simplify the operation of the neural network model and improve the data processing efficiency of the neural network model.

[0005] In a first aspect, a neural network model quantization method is provided. The method comprises: obtaining an original neural network model, the original neural network model comprising a first operator, a second operator, and a first operation module, the first operator and the second operator being used to perform the same type of operation, and the first operation module being used to perform a first operation on an output of the first operator and an output of the second operator; determining a data quantization parameter according to a range of first training input data and a range of second training input data, the first training input data being input data of the first operator, and the second training input data being input data of the second operator; and determining a quantized neural network model according to the original neural network model, the quantized neural network model comprising a quantization module, a third operator, a fourth operator, and a second operation module, the quantization module being used to quantize first input data of the third operator and second input data of the fourth operator using the data quantization parameter, the third operator being a quantized first operator, the fourth operator being a quantized second operator, and the second operation module being used to perform the first operation.

[0006] The data quantization parameter is determined according to the numerical range of the input data of the first operator and the second operator in the original neural network model, and is used to quantize the input data of the third operator and the fourth operator in the quantized neural network model.

[0007] The quantized neural network model can use the same data quantization parameter to quantize the data input to two different operators, so that the processing results of the third operator and the fourth operator correspond to the same quantization parameter, and the third operation can be directly performed on the processing results of the third operator and the fourth operator without the need for reverse quantization of the processing results of the third operator and the fourth operator before the third operation, thereby simplifying the operation of the quantized neural network model and improving the data processing efficiency of the neural network model.

[0008] The data quantization parameter used to quantize the input data of the third operator and the fourth operator is determined according to the numerical range of the data processed by the first operator and the second operator, thereby improving the accuracy of the processing results of the quantized data of the third operator and the fourth operator, reducing the influence of the quantized neural network model on the accuracy of the data processing results, and improving the data processing efficiency of the neural network model.

[0009] With reference to the first aspect, in some possible implementation manners, the method further includes: obtaining preset training output data corresponding to a training input data set, the training input data set including the first training input data and the second training input data; quantizing the first training input data and the second training input data respectively by using the data quantization parameter; processing the quantized first training input data and the quantized second training input data by using the quantized neural network model to obtain actual training output data; adjusting the data quantization parameter according to a difference between the actual training output data and the preset training output data, to minimize the difference; and the quantization module is configured to quantize the first input data of the third operator and the second input data of the fourth operator respectively by using the adjusted data quantization parameter.

[0010] The preset training output data can be manually set. The preset training output data can also be obtained by processing the first training input data and the second training input data by using the original neural network model. For example, the preset training output data can be the output of the operation module.

[0011] Since the data quantization parameter is adjusted in a manner of minimizing the difference between the actual training output data of the quantized neural network model and the preset training output data corresponding to the data, the adjusted data quantization parameter can make the result of processing the quantized data by the third operator and the fourth operator have high accuracy. In this way, the data processing efficiency of the neural network model is improved, and the influence of the quantized neural network model on the accuracy of the data processing result is reduced.

[0012] With reference to the first aspect, in some possible implementation manners, the method further includes: determining an operator quantization parameter according to a parameter range of the first operator and a parameter range of the second operator; quantizing the parameter of the first operator by using the operator quantization parameter to obtain the parameter of the third operator; and quantizing the parameter of the second operator by using the operator quantization parameter to obtain the parameter of the fourth operator.

[0013] Since the operator quantization parameter is determined according to the parameter range of the first operator and the parameter range of the second operator, the quantized neural network model can improve the data processing efficiency while reducing the influence on the accuracy and precision of the data processing result.

[0014] In some possible implementation manners, in combination with the first aspect, the quantized neural network model further includes a compression module, the compression module is configured to compress the output of the third operator and the output of the fourth operator respectively according to an offset parameter, the offset parameter is used to indicate the position of the highest bit in the compressed data before the compression, and the second operation module is configured to perform the first operation on the compressed data; the method further includes: quantizing the first training input data and the second training input data respectively by using the data quantization parameter; processing the quantized first training input data by using the third operator, the third operator outputs first training operation data; processing the quantized second training input data by using the fourth operator, the fourth operator outputs second training operation data; and determining the offset parameter according to the number of effective bits of the first training operation data and the number of effective bits of the second training operation data.

[0015] The same offset parameter is used to compress the outputs of the third operator and the fourth operator, which can improve the data processing efficiency of the neural network model. Since the offset parameter is determined according to the number of effective bits of the intermediate operation result obtained by processing the training input data by using the quantized neural network model, when the quantized neural network model processes data, the offset parameter is used to compress the intermediate operation result, which can reduce the influence on the accuracy and precision of the final data processing result.

[0016] In the second aspect, a data processing method is provided, the method includes: obtaining a quantized neural network model, the quantized neural network model is obtained by quantizing an original neural network model, the original neural network model includes a first operator, a second operator and a first operation module, the first operator and the second operator are used to perform the same type of operation, and the first operation module is used to perform a first operation on the output of the first operator and the output of the second operator; processing first input data of the third operator and second input data of the fourth operator by using the quantized neural network model, the quantized neural network model includes a quantization module, a first operator, a second operator and a second operation module, the quantization module is used to quantize the first input data and the second input data respectively by using a data quantization parameter, the second operation module is used to perform the first operation, the third operator is a quantized first operator, the fourth operator is a quantized second operator, and the data quantization parameter is determined according to the range of first training input data of the first operator and the range of second training input data of the second operator.

[0017] With reference to the second aspect, in some possible implementations, the data quantization parameter is obtained by adjusting an initial data quantization parameter, the adjusting is performed to minimize a difference between actual training output data and preset training output data, the initial data quantization parameter is determined according to a range of the first training input data and a range of the second training input data, the preset training output data corresponds to a training input data group, the training input data group includes the first training input data and the second training input data, the actual training output data is obtained by processing the first training input data and the second training input data by using the quantized neural network model, and the quantization module is configured to quantize the first training input data and the second training input data by using the initial data quantization parameter, respectively.

[0018] With reference to the second aspect, in some possible implementations, a parameter of the third operator is obtained by quantizing a parameter of the first operator by using an operator quantization parameter, a parameter of the fourth operator is obtained by quantizing a parameter of the second operator by using the operator quantization parameter, and the operator quantization parameter is determined according to a parameter range of the first operator and a parameter range of the second operator.

[0019] With reference to the second aspect, in some possible implementations, the quantized neural network model further includes a compression module, the compression module is configured to compress an output of the third operator and an output of the fourth operator by using an offset parameter, the offset parameter is used to indicate a position of a highest bit in data after the compression in data before the compression, and the second operation module is configured to perform the first operation on the compressed data; and the offset parameter is determined according to a number of effective bits of first training operation data and a number of effective bits of second training operation data, the first training operation data is obtained by processing the first training input data quantized by using the data quantization parameter by using the third operator, and the second training operation data is obtained by processing the second training input data quantized by using the data quantization parameter by using the fourth operator.

[0020] In a third aspect, a neural network model quantization apparatus is provided, and the apparatus includes a storage module and a processing module. The storage module is configured to store a program. When the program is executed in the processing module, the processing module is configured to: obtain an original neural network model, the original neural network model including a first operator, a second operator, and a first operation module, the first operator and the second operator being configured to perform the same type of operation, and the first operation module being configured to perform a first operation on an output of the first operator and an output of the second operator; determine a data quantization parameter according to a range of first training input data and a range of second training input data, the first training input data being input data of the first operator, and the second training input data being input data of the second operator; and determine a quantized neural network model according to the original neural network model, the quantized neural network model including a quantization module, a third operator, a fourth operator, and a second operation module, the quantization module being configured to quantize first input data of the third operator and second input data of the fourth operator using the data quantization parameter, the third operator being a quantized first operator, the fourth operator being a quantized second operator, and the second operation module being configured to perform the first operation.

[0021] With reference to the third aspect, in some possible implementation manners, the processing module is further configured to: obtain preset training output data corresponding to a training input data set, the training input data set including the first training input data and the second training input data; quantize the first training input data and the second training input data using the data quantization parameter; process the quantized first training input data and the quantized second training input data using the quantized neural network model to obtain actual training output data; and adjust the data quantization parameter according to a difference between the actual training output data and the preset training output data, so as to minimize the difference; and the quantization module is configured to quantize the first input data of the third operator and the second input data of the fourth operator using the adjusted data quantization parameter.

[0022] With reference to the third aspect, in some possible implementation manners, the processing module is further configured to: determine an operator quantization parameter according to a parameter range of the first operator and a parameter range of the second operator; quantize a parameter of the first operator using the operator quantization parameter to obtain a parameter of the third operator; and quantize a parameter of the second operator using the operator quantization parameter to obtain a parameter of the fourth operator.

[0023] In some possible implementation manners, in combination with the third aspect, the quantized neural network model further includes a compression module, the compression module is configured to compress outputs of the third operator and the fourth operator respectively according to offset parameters, the offset parameters are used to indicate positions of highest bit positions in the compressed data before the compression, and the second operation module is configured to perform the first operation on the compressed data; the processing module is further configured to quantize the first training input data and the second training input data respectively by using the data quantization parameters; the processing module is further configured to process the quantized first training input data by using the third operator, and the third operator outputs first training operation data; the processing module is further configured to process the quantized second training input data by using the fourth operator, and the fourth operator outputs second training operation data; and the processing module is further configured to determine the offset parameters according to the number of significant bits of the first training operation data and the number of significant bits of the second training operation data.

[0024] In a fourth aspect, a data processing apparatus is provided, including a storage module and a processing module, the storage module is configured to store a program; when the program runs in the processing module, the processing module is configured to: obtain a quantized neural network model, the quantized neural network model is obtained by quantizing an original neural network model, the original neural network model includes a first operator, a second operator and a first operation module, the first operator and the second operator are used to perform the same type of operation, and the first operation module is used to perform a first operation on an output of the first operator and an output of the second operator; process first input data of the third operator and second input data of the fourth operator by using the quantized neural network model, the quantized neural network model includes a quantization module, a first operator, a second operator and a second operation module, the quantization module is configured to quantize the first input data and the second input data respectively by using data quantization parameters, the second operation module is configured to perform the first operation, the third operator is a quantized first operator, the fourth operator is a quantized second operator, and the data quantization parameters are determined according to a range of first training input data of the first operator and a range of second training input data of the second operator.

[0025] In some possible implementation manners, the data quantization parameter is obtained by adjusting an initial data quantization parameter, the adjusting is performed to minimize a difference between actual training output data and preset training output data, the initial data quantization parameter is determined according to a range of the first training input data and a range of the second training input data, the preset training output data corresponds to a training input data group, the training input data group includes the first training input data and the second training input data, the actual training output data is obtained by processing the first training input data and the second training input data by using the quantized neural network model, and the quantization module is configured to quantize the first training input data and the second training input data by using the initial data quantization parameter.

[0026] In some possible implementation manners, a parameter of the third operator is obtained by quantizing a parameter of the first operator by using an operator quantization parameter, and a parameter of the fourth operator is obtained by quantizing a parameter of the second operator by using the operator quantization parameter, and the operator quantization parameter is determined according to a parameter range of the first operator and a parameter range of the second operator.

[0027] In some possible implementation manners, the quantized neural network model further includes a compression module, the compression module is configured to compress an output of the third operator and an output of the fourth operator by using an offset parameter, the offset parameter is used to indicate a position of a highest bit in data after the compression in data before the compression, and the second operation module is configured to perform the first operation on the compressed data, and the offset parameter is determined according to a number of effective bits of first training operation data and a number of effective bits of second training operation data, the first training operation data is obtained by processing the first training input data quantized by using the data quantization parameter by using the third operator, and the second training operation data is obtained by processing the second training input data quantized by using the data quantization parameter by using the fourth operator.

[0028] In a fifth aspect, an electronic device is provided, including a memory and a processor, the memory is configured to store program instructions, and when the program instructions are executed in the processor, the processor is configured to execute the method in the first aspect or the second aspect.

[0029] The processor in the third aspect can include a central processing unit (CPU), or a combination of a CPU and a neural network operation processor.

[0030] In a sixth aspect, a computer readable medium storing program code for execution by an apparatus is provided, the program code comprising instructions for performing the method of the first aspect or any of the implementation forms of the first aspect.

[0031] In a seventh aspect, a computer program product containing instructions that, when executed on a computer, cause the computer to perform the method of the first aspect or any of the implementation forms of the first aspect is provided.

[0032] In an eighth aspect, a chip is provided, the chip comprising a processor and a data interface, the processor being configured to read instructions stored on a memory via the data interface and perform the method of the first aspect or any of the implementation forms of the first aspect.

[0033] Optionally, as an implementation form, the chip can further comprise a memory, the memory storing instructions, the processor being configured to execute the instructions stored on the memory, the instructions, when executed, causing the processor to perform the method of the first aspect or any of the implementation forms of the first aspect.

[0034] The chip can specifically be a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 A structural schematic diagram of a system architecture provided by an embodiment of the present application.

[0036] Figure 2 A structural schematic diagram of a convolutional neural network provided by an embodiment of the present application.

[0037] Figure 3 A structural schematic diagram of another convolutional neural network provided by an embodiment of the present application.

[0038] Figure 4 A hardware structural schematic diagram of a chip provided by an embodiment of the present application.

[0039] Figure 5 A structural schematic diagram of a system architecture provided by an embodiment of the present application.

[0040] Figure 6 A schematic structural diagram of a neural network model quantization apparatus provided by an embodiment of the present application.

[0041] Figure 7is a schematic flow chart of a neural network model quantization method provided by an embodiment of the present application.

[0042] Figure 8 is a schematic flow chart of another neural network model quantization method provided by an embodiment of the present application.

[0043] Figure 9 is a schematic structural diagram of a data processing system provided by an embodiment of the present application.

[0044] Figure 10 is a schematic diagram of data before and after compression provided by an embodiment of the present application.

[0045] Figure 11 is a schematic flow chart of a data processing method provided by an embodiment of the present application.

[0046] Figure 12 is a schematic flow chart of another data processing method provided by an embodiment of the present application.

[0047] Figure 13 is a schematic flow chart of a processing structure identification method provided by an embodiment of the present application.

[0048] Figure 14 is a schematic structural diagram of a neural network model quantization apparatus provided by an embodiment of the present application.

[0049] Figure 15 is a schematic structural diagram of a data processing apparatus provided by an embodiment of the present application.

[0050] Figure 16 is a hardware structural schematic diagram of a data processing apparatus of an embodiment of the present application.

[0051] Figure 17 is a hardware structural schematic diagram of a neural network model quantization apparatus of an embodiment of the present application. DETAILED DESCRIPTION

[0052] The technical solutions in the present application will be described below with reference to the drawings. The technical solutions in the present application will be described below with reference to the drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0053] Since the embodiments of the present application involve the application of a large number of neural networks, in order to facilitate understanding, the related terms and concepts of the neural networks that may be involved in the embodiments of the present application will be introduced first.

[0054] (1) Neural network:

[0055] The neural network can be composed of neural units, which can refer to a neuron-like unit that takes x s and an intercept 1 as inputs, and the output of the operation unit can be:

[0056]

[0057] where s = 1, 2, … n, n is a natural number greater than 1, W s is the weight of x s , b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce non-linear characteristics into the neural network to convert the input signal in the neural unit into an output signal. The output signal of the activation function can be used as the input of the next layer. For example, the activation function can be a ReLU, tanh or sigmoid function.

[0058] The neural network is a network formed by connecting a plurality of the above single neural units, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, and the local receptive field can be a region composed of a plurality of neural units.

[0059] (2) Deep neural network:

[0060] The deep neural network (DNN) is also called a multi-layer neural network, which can be understood as a neural network with multiple hidden layers. According to the position of different layers, the neural network inside the DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally, the first layer is the input layer, the last layer is the output layer, and the number of layers in between is the hidden layer. The layers are fully connected, that is, any neuron in the i-th layer is connected to any neuron in the i+1-th layer.

[0061] Although the DNN looks very complex, in terms of the work of each layer, it is actually not complex. In simple terms, it is expressed as the following linear relationship expression: wherein, is the input vector, is the output vector, is the offset vector, W is the weight matrix (also called coefficient), is the activation function. Each layer only performs the following simple operation on the input vector to obtain the output vector . Due to the large number of layers of the DNN, the coefficients W and the offset vector The number of these parameters is also relatively large. These parameters in the DNN are defined as follows: the coefficient W For example: suppose in a three-layer DNN, the linear coefficient from the 4th neuron of the second layer to the 2nd neuron of the third layer is defined as The superscript 3 represents the layer number where the coefficient W is located, and the subscript corresponds to the output third layer index 2 and the input second layer index 4.

[0062] In summary, the coefficient from the kth neuron of the L−1th layer to the jth neuron of the Lth layer is defined as .

[0063] It should be noted that the input layer has no W parameters. In a deep neural network, more hidden layers enable the network to better depict complex situations in the real world. In theory, the more parameters a model has, the higher its complexity and the greater its "capacity", which means it can complete more complex learning tasks. Training a deep neural network is a process of learning the weight matrix, and the ultimate goal is to obtain the weight matrix of all layers of the trained deep neural network (the weight matrix formed by many layers of vectors W).

[0064] (3) Convolutional Neural Network

[0065] A convolutional neural network (CNN) is a deep neural network with a convolutional structure. The convolutional neural network includes a feature extractor composed of a convolutional layer and a subsampling layer, which can be regarded as a filter. The convolutional layer refers to the neural layer that performs convolution processing on the input signal in the convolutional neural network. In the convolutional layer of the convolutional neural network, a neuron can be connected only to part of the adjacent layer neurons. A convolutional layer usually contains several feature planes, each of which can be composed of some rectangularly arranged neural units. The neural units of the same feature plane share weights, and the shared weights are the convolution kernel. Shared weights can be understood as being independent of the way and position of extracting data information. The convolution kernel can be initialized in the form of a random size matrix, and the convolution kernel can obtain reasonable weights through learning in the training process of the convolutional neural network. In addition, the direct benefit of shared weights is to reduce the connections between layers of the convolutional neural network, while also reducing the risk of overfitting.

[0066] (4) Loss Function

[0067] In the process of training a deep neural network, because the output of the deep neural network is expected to be as close as possible to the value that is truly intended to be predicted, the weight vector of each layer of the neural network can be updated according to the difference between the predicted value of the current network and the truly intended target value, after comparing the two (of course, before the first update, there is usually an initialization process, that is, the parameters of each layer of the deep neural network are pre-configured), for example, if the predicted value of the network is too high, adjust the weight vector to make it predict lower, and keep adjusting until the deep neural network can predict the truly intended target value or a value very close to the truly intended target value. Therefore, it is necessary to define "how to compare the difference between the predicted value and the target value", which is the loss function or the objective function, which is an important equation for measuring the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, and then the training of the deep neural network becomes a process of trying to minimize the loss.

[0068] (5) Back propagation algorithm

[0069] The neural network can use the back propagation (BP) algorithm to correct the size of the parameters in the initial neural network model in the training process, so that the reconstruction error loss of the neural network model becomes smaller and smaller. Specifically, the forward propagation of the input signal will produce an error loss, and the initial neural network model parameters are updated by back propagating the error loss information, so as to make the error loss converge. The back propagation algorithm is a back propagation movement dominated by error loss, aiming to obtain the optimal parameters of the neural network model, such as the weight matrix.

[0070] (6) Data processing

[0071] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision making and the like.

[0072] Among them, machine learning and deep learning can model, extract, preprocess, train and the like of symbolic and formalized intelligent information of data.

[0073] Reasoning refers to the process of simulating human intelligent reasoning methods in a computer or intelligent system, using formalized information to perform machine thinking and solve problems according to reasoning control strategies, and the typical function is search and matching.

[0074] Decision making refers to the process of decision making after intelligent information is reasoned, which usually provides functions such as classification, sorting, prediction and the like.

[0075] For example, Figure 1As shown, the embodiment of the present application provides a system architecture 100. In Figure 1 The data collection device 160 is configured to collect training data. For the data processing method of the embodiment of the present application, the training data can include a plurality of training input data and a training identifier corresponding to each training input data.

[0076] After the training data is collected, the data collection device 160 stores the training data in the database 130, and the training device 120 trains the target model / rule 101 based on the training data maintained in the database 130.

[0077] The training device 120 trains the target model / rule 101 based on the training data, processes the input training input data, compares the output result with the training identifier corresponding to the training input data, until the difference between the output result of the training device 120 and the training identifier is less than a certain threshold, thereby completing the training of the target model / rule 101.

[0078] The target model / rule 101 described above can be used to implement the data processing method of the embodiment of the present application. The target model / rule 101 in the embodiment of the present application can be a neural network. It should be noted that in actual application, the training data maintained in the database 130 can not all come from the collection of the data collection device 160, but can also be received from other devices. In addition, it should be noted that the training device 120 can not completely train the target model / rule 101 based on the training data maintained in the database 130, but can also obtain the training data from the cloud or other places for model training. The above description should not be regarded as a limitation of the embodiment of the present application.

[0079] The target model / rule 101 trained by the training device 120 can be applied to different systems or devices, such as the execution device 110 shown in the embodiment of the present application. Figure 1 The execution device 110 can be a terminal, such as a mobile phone terminal, a tablet computer, a notebook computer, an augmented reality (AR) AR / virtual reality (VR), a vehicle terminal, etc., and can also be a server or a cloud, etc. In Figure 1 The execution device 110 is configured with an input / output (I / O) interface 112 for data interaction with external devices. A user can input data to the I / O interface 112 through the client device 140, and the input data in the embodiment of the present application can include the data to be processed input by the client device.

[0080] The preprocessing modules 113 and 114 are configured to perform preprocessing on input data (e.g., data to be processed) received by the I / O interface 112. In some embodiments, the preprocessing modules 113 and 114 can be omitted (only one of them can be used) and the input data can be directly processed by the computing module 111.

[0081] During the preprocessing of the input data by the execution device 110 or the processing performed by the computing module 111 of the execution device 110, the execution device 110 can call data and codes in the data storage system 150 for the corresponding processing, and the data and instructions obtained by the corresponding processing can be stored in the data storage system 150.

[0082] Finally, the I / O interface 112 returns the processing result (e.g., the processing result of the data) to the client device 140, thereby providing the user with the result.

[0083] It is worth noting that the training device 120 can generate a corresponding target model / rule 101 based on different training data for different targets or tasks, and the corresponding target model / rule 101 can be used to achieve the above targets or complete the above tasks, thereby providing the user with the required result.

[0084] In the case shown in Figure 1 , the user can manually input data, which can be operated through the interface provided by the I / O interface 112. In another case, the client device 140 can automatically send input data to the I / O interface 112. If the user needs to authorize the client device 140 to automatically send input data, the user can set the corresponding permission in the client device 140. The user can view the result output by the execution device 110 in the client device 140, and the specific presentation form can be a specific way such as display, sound, and action. The client device 140 can also be used as a data collection terminal to collect the input data input into the I / O interface 112 and the output result output from the I / O interface 112 as new sample data and store them in the database 130. Of course, the input data input into the I / O interface 112 and the output result output from the I / O interface 112 can be directly stored in the database 130 as new sample data without the collection by the client device 140.

[0085] It is worth noting that Figure 1 The system architecture shown in the figure is only an example provided by the embodiments of the present application, and the positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in Figure 1In some embodiments, the data storage system 150 is an external memory to the execution device 110, and in other embodiments, the data storage system 150 can also be placed in the execution device 110.

[0086] As shown in FIG. 1, the target model / rule 101 is trained by the training device 120, and the target model / rule 101 can be a neural network in the present application, and specifically, the neural network used in the present application can be a CNN, a deep convolutional neural network (DCNN), a recurrent neural network (RNN), etc. Figure 1

[0087] Since the CNN is a very common neural network, the structure of the CNN will be introduced in detail below. Figure 2 As described above, the CNN is a deep neural network with a convolution structure, and is a deep learning architecture, which means that the CNN is a feed-forward artificial neural network that can respond to data input into the CNN.

[0088] The structure of the neural network used in the data processing method of the present application can be as shown in FIG. 2. In the present application, the CNN 200 can include an input layer 210, a convolution layer / pooling layer 220 (where the pooling layer is optional), and a neural network layer 230. The input layer 210 can obtain the data to be processed, and pass the obtained data to the convolution layer / pooling layer 220 and the neural network layer 230 for processing, and the processing result of the data can be obtained. Figure 2 Figure 2 Figure 2

[0089] Convolution layer / pooling layer 220:

[0090] Convolution layer:

[0091] As shown in FIG. 2, the input layer 210 can obtain the data to be processed, and pass the obtained data to the convolution layer / pooling layer 220 and the neural network layer 230 for processing, and the processing result of the data can be obtained. Figure 2 ​​​​The convolutional / pooling layer 220 shown may include layers as in Examples 221-226. For instance, in one implementation, layer 221 is a convolutional layer, layer 222 is a pooling layer, layer 223 is a convolutional layer, layer 224 is a pooling layer, layer 225 is a convolutional layer, and layer 226 is a pooling layer; in another implementation, layers 221 and 222 are convolutional layers, layer 223 is a pooling layer, layers 224 and 225 are convolutional layers, and layer 226 is a pooling layer. That is, the output of the convolutional layer can be used as the input to a subsequent pooling layer, or as the input to another convolutional layer to continue the convolution operation.

[0092] The following section will use convolutional layer 221 as an example to introduce the internal working principle of a convolutional layer.

[0093] Convolutional layer 221 can include many convolution operators, also known as kernels. In data processing, a convolution operator acts as a filter that extracts specific information from the input data matrix. A convolution operator can essentially be a weight matrix, which is usually predefined.

[0094] The weight values ​​in these weight matrices need to be obtained through extensive training in practical applications. The weight matrices formed by the weight values ​​obtained through training can be used to extract information from the input data, thereby enabling the convolutional neural network 200 to make correct predictions.

[0095] When a convolutional neural network 200 has multiple convolutional layers, the initial convolutional layers (e.g., 221) tend to extract more general features, which can also be called low-level features. As the depth of the convolutional neural network 200 increases, the features extracted by later convolutional layers (e.g., 226) become more and more complex, such as high-level semantic features. Features with higher semantic levels are more suitable for the problem to be solved.

[0096] Pooling layer:

[0097] Because it is often necessary to reduce the number of training parameters, pooling layers are often introduced periodically after convolutional layers, such as... Figure 2 In the example of 220, layers 221-226 can be a convolutional layer followed by a pooling layer, or multiple convolutional layers followed by one or more pooling layers. The sole purpose of pooling layers in data processing is to reduce the spatial size of the data.

[0098] Neural network layer 230:

[0099] After the processing of the convolutional / pooling layer 220, the convolutional neural network 200 is still not sufficient to output the required output information. As mentioned before, the convolutional / pooling layer 220 only extracts features and reduces the parameters brought by the input data. However, in order to generate the final output information (the required class information or other related information), the convolutional neural network 200 needs to utilize the neural network layer 230 to generate an output of the number of required classes or a group of required classes. Therefore, the neural network layer 230 can include multiple hidden layers (such as 231, 232, and 23n as shown) and an output layer 240, and the parameters included in the multiple hidden layers can be pre-trained according to the related training data of a specific task type, which can include recognition, classification, and the like. Figure 2

[0100] After the multiple hidden layers in the neural network layer 230, that is, the last layer of the entire convolutional neural network 200 is the output layer 240, which has a loss function similar to the classification cross-entropy, and is specifically used to calculate the prediction error. Once the forward propagation (such as the propagation from 210 to 240) of the entire convolutional neural network 200 is completed, the back propagation (such as the propagation from 240 to 210) will start to update the weight values and biases of the layers mentioned above, so as to reduce the loss of the convolutional neural network 200 and the error between the result output by the convolutional neural network 200 through the output layer and the ideal result. Figure 2 Figure 2

[0101] The neural network structure specifically used by the data processing method of the embodiments of the present application can be as shown in Figure 3 . In Figure 3 , the convolutional neural network (CNN) 200 can include an input layer 210, a convolutional / pooling layer 220 (in which the pooling layer is optional), and a neural network layer 230. Compared with Figure 2 , the multiple convolutional / pooling layers in the convolutional / pooling layer 220 in Figure 3 are parallel, and the features extracted by the multiple convolutional / pooling layers are respectively input to the neural network layer 230 for processing.

[0102] It should be noted that Figure 2 and Figure 3 the convolutional neural network is only an example of two possible convolutional neural networks of the data processing method of the embodiments of the present application, and in specific applications, the convolutional neural network used by the data processing method of the embodiments of the present application can also exist in the form of other network models.

[0103] Figure 4 A chip hardware structure provided by the embodiments of the present application includes a neural network processor 50. The chip can be arranged in a​​​Figure 1 The execution device 110 shown is used to complete the calculation work of the calculation module 111. The chip can also be arranged in the training device 120 shown Figure 1 The training device 120 shown is used to complete the training work of the training device 120 and output the target model / rule 101. As shown in the figure Figure 2 The algorithm of each layer of the convolutional neural network shown Figure 3 The algorithm of each layer of the convolutional neural network shown Figure 4 The chip shown can be implemented.

[0104] The neural network processor NPU 50 is mounted as a coprocessor on a host central processing unit (CPU), and tasks are allocated by the host CPU. The core part of the NPU is the operation circuit 503, and the controller 504 controls the operation circuit 503 to extract data in the memory (weight memory or input memory) and perform operations.

[0105] In some implementations, the operation circuit 503 includes multiple processing units (PEs) inside. In some implementations, the operation circuit 503 is a two-dimensional systolic array. The operation circuit 503 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 503 is a general matrix processor.

[0106] For example, assuming there is an input matrix A, a weight matrix B, and an output matrix C. The operation circuit takes the corresponding data of matrix B from the weight memory 502 and caches it on each PE in the operation circuit. The operation circuit takes the matrix A data from the input memory 501 and performs matrix operations with matrix B to obtain partial results or final results of the matrix, which are saved in the accumulator 508.

[0107] The vector calculation unit 507 can further process the output of the operation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. For example, the vector calculation unit 507 can be used for network calculation of non-convolutional / non-FC layers in the neural network, such as pooling, batch normalization, local response normalization, etc.

[0108] In some implementations, the vector computation unit 507 can store the processed output vector to the unified buffer 506. For example, the vector computation unit 507 can apply a non-linear function to the output of the arithmetic circuit 503, e.g., a vector of accumulated values, to generate activation values. In some implementations, the vector computation unit 507 generates normalized values, merged values, or both. In some implementations, the processed output vector can be used as activation input to the arithmetic circuit 503, e.g., for use in a subsequent layer in a neural network.

[0109] The unified memory 506 is used to store input data and output data.

[0110] The weight data is transferred from the external memory to the input memory 501 and / or the unified memory 506, from the external memory to the weight memory 502, and from the unified memory 506 to the external memory by a direct memory access controller (DMAC) 505.

[0111] A bus interface unit (BIU) 510 is used to interact with the main CPU, the DMAC, and the instruction fetch buffer 509 via a bus.

[0112] An instruction fetch buffer 509 connected to the controller 504 is used to store instructions used by the controller 504.

[0113] The controller 504 is used to invoke the instructions cached in the instruction fetch buffer 509 to control the operation of the arithmetic accelerator.

[0114] Generally, the unified memory 506, the input memory 501, the weight memory 502, and the instruction fetch buffer 509 are on-chip memories, and the external memory is a memory external to the NPU. The external memory can be a double data rate synchronous dynamic random access memory (DDR SDRAM), a high bandwidth memory (HBM), or other readable and writable memories.

[0115] wherein, Figure 2 and Figure 3 The operations of the layers in the convolutional neural network shown in FIG. 7 can be performed by the arithmetic circuit 503 or the vector computation unit 507.

[0116] The execution device 110 in the above Figure 1 can execute each step of the data processing method of the embodiments of the present application, Figure 2 and Figure 3 the CNN model shown in Figure 4 and the chip shown in can also be used to execute each step of the data processing method of the embodiments of the present application. The method of training the neural network of the embodiments of the present application and the data processing method of the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.

[0117] As shown in Figure 5 , the embodiments of the present application provide a system architecture 300. The system architecture includes a local device 301, a local device 302, and an execution device 110 and a data storage system 150, wherein the local device 301 and the local device 302 are connected with the execution device 110 through a communication network.

[0118] The execution device 110 can be implemented by one or more servers. Optionally, the execution device 110 can be used in cooperation with other computing devices, such as data storage, routers, load balancers, etc. The execution device 110 can be arranged on one physical site, or distributed on multiple physical sites. The execution device 110 can use data in the data storage system 150, or call program code in the data storage system 150 to implement the data processing method of the embodiments of the present application.

[0119] Users can operate their respective user devices (such as the local device 301 and the local device 302) to interact with the execution device 110. Each local device can represent any computing device, such as a personal computer, a computer workstation, a smartphone, a tablet computer, a smart camera, a smart car or other types of cellular phones, a media consumption device, a wearable device, a set-top box, a game console, etc.

[0120] Each user's local device can interact with the execution device 110 through a communication network of any communication mechanism / communication standard, which can be a wide area network, a local area network, a point-to-point connection, etc., or any combination thereof.

[0121] In an implementation manner, the local device 301 and the local device 302 obtain the related parameters of the target neural network from the execution device 110, deploy the target neural network on the local device 301 and the local device 302, and use the target neural network for data classification or recognition, etc.

[0122] In another implementation, the target neural network can be directly deployed on the execution device 110, and the execution device 110 obtains to-be-processed data from the local device 301 and the local device 302, and classifies or performs other types of data processing on the to-be-processed data according to the target neural network.

[0123] The execution device 110 described above can also be a cloud device, in which case the execution device 110 can be deployed in the cloud; or the execution device 110 described above can also be a terminal device, in which case the execution device 110 can be deployed on the user terminal side, and the embodiments of the present application do not limit this.

[0124] At present, neural network models are widely used in many fields such as images, videos, and speech, and have shown the ability to surpass traditional methods. However, the neural network model itself has a large amount of computation and a large number of parameters, which brings great challenges to the deployment of neural networks on terminal devices.

[0125] Model quantization is used to quantize the parameters of the operators in the neural network model and to quantize the input data. Quantizing the parameters of the operators can optimize the size of the operators and reduce the resources occupied by the operators. On this basis, quantizing the input data of the operators can convert the floating-point operations of the operators into fixed-point operations, improve the inference speed, and reduce power consumption. Compared with a neural network model represented by single-precision floating-point numbers (usually 32 bits), a quantized neural network model obtained by 8-bit quantization can reduce the storage space occupied by each parameter to one-fourth and process data at a better inference speed.

[0126] In the process of model quantization of the operators, in order to improve the data processing accuracy of the quantized operators, the operator quantization parameters need to be determined according to the range of the parameters of each operator, and the data quantization parameters need to be determined according to the range of the input data. The differences in the ranges of the parameters of different operators and the ranges of the input data result in differences in the data quantization parameters and / or the operator quantization parameters corresponding to each operator. Before the data processing results of a plurality of quantized operators are operated, the data processing results need to be respectively dequantized to ensure the accuracy of the calculation results. In an NPU, the number of processing units used for dequantization operation is limited, which limits the operation rate of dequantization and makes the overall processing efficiency low and the performance poor. The dequantization parameters used when each operator's data processing result is dequantized can be determined according to the data quantization parameters and the operator quantization parameters corresponding to the operator. In order to solve the above problems, the embodiments of the present application provide a neural network model quantization device, which can reduce the number of dequantization operations that the quantized neural network model needs to perform subsequently and improve the overall processing performance.

[0127] Figure 6This is a schematic structural diagram of a neural network model quantization device provided in an embodiment of this application. The neural network model quantization device 1300 can be located in Figure 1 The training device 120 or other device shown includes a neural network model quantization device 1300, which comprises a data quantization parameter generation model 1310 and an operator quantization model 1320. The neural network model quantization device 1300 is used to quantize the original neural network model. The original neural network model includes a first operator, a second operator, and a first operation module. The first and second operators are used to perform the same type of operation. The first operation module is used to perform a first operation on the output of the first operator and the output of the second operator. The quantization parameter generation model 1310 is used to generate data quantization parameters based on the range of data in the training input data set. The training input data set includes the first training input data of the first operator and the second training input data of the second operator. The operator quantization model 1320 is used to quantize the operation units in the original neural network model, such as the first operator and the second operator. Based on the data quantization parameters and the quantized operation units, a quantized neural network model can be obtained.

[0128] The quantized neural network model includes a quantization module, a quantized first operator, a quantized second operator, and a second operation module. The quantization module quantizes the first and second input data using data quantization parameters. The second operation module corresponds to the first operation module in the original neural network model and performs the first operation on the first and second operation data. The first operation data is obtained by the quantized first operator operating on the quantized first input data. The second operation data is obtained by the quantized second operator operating on the quantized second input data.

[0129] Figure 7 This is a schematic flowchart illustrating a neural network model quantization method provided in an embodiment of this application. This neural network model quantization method can be... Figure 1 The training is performed using the training device 120 or other devices shown.

[0130] S1101, Obtain the original neural network model, which includes a first operator, a second operator, and a first operation module. The first and second operators are used to perform the same type of operation, and the first operation module is used to perform a first operation on the output of the first operator and the output of the second operator. The original neural network model can be obtained from messages sent by other devices. Alternatively, it can be obtained from memory. The original neural network model can be a CNN, etc.

[0131] The first operation and the second operation are operations of the same type, i.e., the first operator and the second operator are operators of the same type. For example, the first operator and the second operator can both be convolutional layers in a CNN or both be fully connected layers. The first operation module can be configured to perform a bitwise operation, such as bitwise addition or bitwise multiplication, on the output of the first operator and the output of the second operator. Generally, the first operation module can be configured to perform a linear operation.

[0132] The neural network model quantization method includes steps S1101 and S1102, and is configured to quantize an original neural network model to obtain a quantized neural network model. The neural network model quantization method can be performed by the training device 120 or other devices shown in FIG. 1. Figure 1

[0133] In S1102, a data quantization parameter is determined according to a range of first training input data and a range of second training input data. The first training input data is input data of the first operator, and the second training input data is input data of the second operator. Specifically, an average data range upper limit can be determined according to a maximum value of a plurality of first training input data of the first operator and a maximum value of a plurality of second training input data of the second operator. An average data range lower limit can be determined according to a minimum value of the plurality of first training input data of the first operator and a minimum value of the plurality of second training input data of the second operator. The data quantization parameter can be determined according to the average data range upper limit and the average data range lower limit.

[0134] The average data range upper limit can be understood as an average of the maximum values of the plurality of first training input data and the maximum values of the plurality of second training input data. The average data range lower limit can be understood as an average of the minimum values of the plurality of first training input data and the minimum values of the plurality of second training input data.

[0135] The average data range upper limit and the average data range lower limit can be updated each time training input data is input. Thus, the calculation of the average data range upper limit and the average data range lower limit is dispersed, and compared with the way of obtaining an average value from a plurality of training input data, the requirement for computing resources can be reduced.

[0136] A weight can be introduced to update the average data range upper limit and the average data range lower limit. The specific way of updating the average data range upper limit and the average data range lower limit can be referred to the description of Figure 8 It should be understood that the weight can be used to adjust the influence of the training input data input into the first operator and / or the second operator on the average data range upper limit and the average data range lower limit as the number of iterations increases.

[0137] ​Generally, the bit number (i.e., the number of bit) of the quantized input data of the operator is a preset value, for example, can be 8 bits. Of course, in some embodiments, the bit number of the quantized input data can also be obtained by manually inputting information, etc. The data quantization parameter can include a scale and an offset. The bit number is one-to-one corresponding to the number of scale in the data quantization parameter. The number of scale can be understood as the maximum value that the bit number of the quantized data can represent, i.e., 2 m -1, m is the bit number of the quantized data. According to the difference between the upper limit of the average data range and the lower limit of the average data range and the number of scale, the scale can be obtained. For example, the parameter scale can be the quotient obtained by dividing the difference between the upper limit of the average data range and the lower limit of the average data range by the number of scale, or the parameter scale can be the quotient obtained by dividing the difference between the upper limit of the average data range and the lower limit of the average data range plus 1 by the number of scale. The offset in the data quantization parameter can be the ratio of the average data range lower limit to the parameter scale.

[0138] In S1103, according to the original neural network model, a quantized neural network model is determined, the quantized neural network model includes a quantization module, a third operator, a fourth operator and a second operation module, the quantization module is used to quantize the first input data of the third operator and the second input data of the fourth operator by using the data quantization parameter, and the second operation module is used to perform the first operation.

[0139] The second operation module can be used to perform the third first operation on the first operation data and the second operation data, the first operation data is obtained by using the third operator to operate the quantized first input data, and the second operation data is obtained by using the fourth operator to operate the quantized second input data.

[0140] That is, the second operation module corresponds to the first operation module in the original neural network model.

[0141] Through S1101 to S1103, according to the numerical range of the input data of the first operator and the second operator in the original neural network model, the data quantization parameter is determined, and the data quantization parameter is used to quantize the input data of the third operator and the fourth operator in the quantized neural network model, respectively. Wherein, the third operator and the first operator are used to perform the same operation, the fourth operator and the second operator are used to perform the same operation, and the types of the operations performed by the above-mentioned first operator and the second operator are the same.

[0142] By S1101 to step S1103, the quantized neural network model can use the same data quantization parameter to quantize the input data of the two different operators, so that the processing results of the third operator and the fourth operator correspond to the same quantization parameter, and the third operation can be directly performed on the processing results of the third operator and the fourth operator, without the need for inverse quantization of the processing results of the third operator and the fourth operator before the third operation, simplifying the operation of the quantized neural network model and improving the data processing efficiency of the neural network model.

[0143] In addition, according to the numerical range of the data processed by the first operator and the second operator respectively, the data quantization parameter for quantizing the input data of the third operator and the fourth operator is determined, which improves the accuracy of the processing results of the third operator and the fourth operator on the quantized data, reduces the influence of the quantized neural network model on the accuracy of the data processing results, and improves the data processing efficiency of the neural network model.

[0144] Further, the preset training output data corresponding to the training input data set can be obtained, and the training input data set includes the first training input data and the second training input data.

[0145] The preset training output data can be manually set. The preset training output data can also be obtained by processing the first training input data and the second training input data by the original neural network model. For example, the preset training output data can be the output of the operation module.

[0146] After S1102, the data quantization parameter can be used to quantize the first training input data and the second training input data respectively. The quantized neural network model can be used to process the quantized first training input data and the quantized second training input data to obtain actual training output data.

[0147] The data quantization parameter can be adjusted according to the difference between the actual training output data and the preset training output data to minimize the difference.

[0148] The quantization module is used to quantize the first input data of the third operator and the second input data of the fourth operator using the adjusted data quantization parameter.

[0149] That is, the input data of the third operator and the input data of the fourth operator can be quantized using the adjusted data quantization parameter.

[0150] Since the adjustment manner of the data quantization parameter is to minimize the difference between the actual training output data of the quantized neural network model and the preset training output data corresponding to the data, the adjusted data quantization parameter can make the result precision of the third operator and the fourth operator in processing the quantized data be higher. While improving the data processing efficiency of the neural network model, the influence of the quantized neural network model on the accuracy of the data processing result is reduced.

[0151] The first operator and the third operator are used to perform the same operation. The second operator and the fourth operator are used to perform the same operation.

[0152] The two operators are used to perform the same operation, which can also be understood as the two operators performing the same operation on the input data. The parameters of the two operators are only different in precision, and the parameters of one operator are obtained by quantizing the parameters of the other operator. By using the quantized operator to process the quantized input data, the calculation amount can be reduced.

[0153] It should be understood that, in order to make the processing results of the third operator and the fourth operator on the input data have comparability, i.e., can be directly operated without the need for dequantization and other processing in subsequent reasoning, the parameters of the third operator and the parameters of the fourth operator can be obtained by quantizing the parameters of the same operator.

[0154] The operator quantization parameter can be determined according to the parameter range of the first operator and the parameter range of the second operator.

[0155] The parameters of the first operator can be quantized by using the operator quantization parameter to obtain the parameters of the third operator, and the parameters of the second operator can be quantized by using the operator quantization parameter to obtain the parameters of the fourth operator.

[0156] Since the operator quantization parameter is determined according to the parameter range of the first operator and the parameter range of the second operator, the quantized neural network model improves the data processing efficiency while reducing the influence on the accuracy and precision of the data processing result.

[0157] In order to further improve the data processing efficiency of the quantized neural network model, the data processing result output by the third operator and the data processing result output by the fourth operator can be compressed.

[0158] The quantized neural network model further includes a compression module, which is configured to compress the output of the third operator and the output of the fourth operator according to an offset parameter to obtain the first operation data and the second operation data. The offset parameter is used to indicate the position of the highest bit in the compressed data in the data before the compression.

[0159] Generally, the first operation data obtained by compressing the data processing result output by the third operator and the second operation data obtained by compressing the data processing result output by the fourth operator have the same number of bits.

[0160] The offset parameter indicates the position of the highest bit in the compressed data in the data before the compression. The output of the third operator and the output of the fourth operator are compressed by the same offset parameter, so that the first operation data and the second operation data have comparability and can be directly operated without the need for inverse quantization and other processing in subsequent reasoning.

[0161] In order to improve the data processing efficiency of the neural network model while reducing the influence on the accuracy and precision of the data processing result, the offset parameter can be determined according to the effective number of bits of the output obtained by processing the quantized first training input data by the third operator and the effective number of bits of the output obtained by processing the quantized second training input data by the fourth operator.

[0162] That is, the first training input data and the second training input data are quantized by the data quantization parameter respectively. The quantized first training input data can be processed by the third operator, and the quantized second training input data can be processed by the fourth operator. The output of the third operator is the first training operation data, and the output of the fourth operator is the second training operation data.

[0163] Then, the offset parameter can be determined according to the effective number of bits of the first training operation data and the effective number of bits of the second training operation data.

[0164] The determination method of the data quantization parameter, the operator quantization parameter and the offset parameter can be referred to the description of Figure 8 .

[0165] Through S1101 to S1103, the quantized neural network model can be obtained. The quantized neural network model can be, for example, the data processing system 600 shown in Figure 9 , or the data processing system 600 can call each operator or module in the quantized neural network model to process data.

[0166] Figure 8 is a schematic flow chart of a neural network model quantization method provided by an embodiment of the present application.

[0167] The neural network model quantization method 800 can also be understood as an optimization method or a further training method of the neural network model. The training method 800 can be executed by the training device 120 shown in Figure 1 or other devices.

[0168] The original neural network model comprises a first operator, a second operator and an operation module. The operation module is configured to operate the output of the first operator and the output of the second operator. The first operator and the second operator are configured to perform the same type of operation.

[0169] In the original neural network model, the parameters of the first operator and the second operator are represented by a format of a floating point number.

[0170] At S810, an operator quantization parameter is determined according to a parameter range of the first operator and a parameter range of the second operator.

[0171] The scale in the operator quantization parameter can be represented as s2:

[0172]

[0173] wherein, is a maximum parameter value of the first operator and the second operator represented by a floating point number, is a minimum parameter value of the first operator and the second operator represented by a floating point number, and a is a number of bits of a quantization result. For data in the format of int8, the value of a is 8.

[0174] The offset in the operator quantization parameter can be represented as o2:

[0175]

[0176] Generally, is a negative number.

[0177] The parameters of the first operator can be quantized by using the operator quantization parameter to obtain a quantized first operator. The parameters of the second operator can be quantized by using the operator quantization parameter to obtain a quantized second operator.

[0178] At S820, a training data set is obtained, the training data set comprising a training input data group and a preset operation result corresponding to the training input data group. Each training input data group comprises a first training input data and a second training input data.

[0179] It should be understood that one preset operation result corresponds to one first training input data and one second training input data. The preset operation result corresponding to the first training input data and the second training input data can be a floating point number representation of an operation result obtained by operating a processing result of the first operator on the first training input data and a processing result of the second operator on the second training input data. Alternatively, the preset operation result corresponding to the first training input data and the second training input data can be artificially set.

[0180] At S830, the data quantization parameter is determined according to the range of the first training input data and the range of the second training input data.

[0181] The training data set can include a plurality of first training input data and a plurality of second training input data. The data quantization parameter can be determined according to the range of each training input data. The training input data is the first training input data or the second training input data. The data quantization parameter can be the average range of the plurality of first training input data and the plurality of second training input data.

[0182] The scale in the data quantization parameter can be expressed as s1:

[0183]

[0184] wherein, is the average maximum value (which can also be understood as the average upper limit of the data range) obtained after the tth iteration, is the average minimum value (which can also be understood as the average lower limit of the data range) obtained after the tth iteration, and m is used to represent the number of bits of the quantization result obtained by quantizing the training input data. For data in the int8 format, the value of m is 8. It should be understood that m is a preset value.

[0185] The average maximum value of the training input data can be expressed as:

[0186]

[0187] wherein, is the average maximum value of the training input data obtained after the (t-1)th iteration, is the maximum value of the input data of the operator obtained by the tth iteration, is updated constantly with the number of iterations, , is a constant. In the process of multiple iterations, the input data of the operator includes the first training input data of the first operator and the second training input data of the second operator. It can be understood as a weight.

[0188] When is greater than 1, the more the number of iterations, the smaller the influence of the maximum value of the training input data on the average upper limit of the data range as the iteration proceeds. When is less than 1, the more the number of iterations, the greater the influence of the maximum value of the training input data on the average upper limit of the data range as the iteration proceeds. When is equal to 1, the influence of the maximum value of each training input data on the average upper limit of the data range is the same. Generally, the value of is slightly greater than 1 to avoid excessive correction of the average upper limit of the data range.

[0189] The average minimum value of the training input data can be expressed as:

[0190]

[0191] wherein, is the average maximum value of the training input data obtained after the t-1th iteration, is the minimum value of the input data of the operator obtained by the tth iteration, is constantly updated with the number of iterations, , is a constant. In the process of multiple iterations, the input data of the operator includes the first training input data of the first operator and the second training input data of the second operator.

[0192] Similarly, when is greater than 1, the more the number of iterations, the smaller the influence of the maximum value of the training input data on the upper limit of the average data range. When is less than 1, the more the number of iterations, the greater the influence of the maximum value of the training input data on the upper limit of the average data range. When is equal to 1, the influence of the maximum value of each training input data on the upper limit of the average data range is the same. Generally, is slightly greater than 1 to avoid excessive correction of the upper limit of the average data range. and may be equal or not equal.

[0193] Before iteration, the parameters , , , may be set. Generally, the parameters may be set to 0. , may be set according to empirical values, for example may be set to 6.

[0194] In S840, the first training operation data and the second training operation data are operated to obtain training output data, wherein the first training operation data is obtained by processing the first training input data quantized by the data quantization parameter by the first operator quantized by the operator quantization parameter, and the second training operation data is obtained by processing the second training input data quantized by the data quantization parameter by the second operator quantized by the operator quantization parameter.

[0195] The first training input data and the second training input data can be processed respectively by using the data quantization parameter to obtain quantized first training input data and quantized second training input data. The quantized first training input data can be input into the quantized first operator to obtain first training operation data. The quantized second training input data can be input into the quantized second operator to obtain second training operation data. Then, the first training operation data and the second training operation data can be operated to obtain training output data.

[0196] The first training operation data can be obtained by converting data output by the quantized first operator. The number of bits of the first training operation data is less than the number of bits of the data output by the quantized first operator.

[0197] The second training operation data can be obtained by converting data output by the quantized second operator. The number of bits of the second training operation data is less than the number of bits of the data output by the quantized second operator.

[0198] For the processing result of the quantized first operator on the quantized first training input data and the processing result of the quantized second operator on the quantized second training input data, the average effective number of bits can be counted.

[0199] The average data range upper limit can be counted to obtain the average effective number of bits.

[0200] After t iterations, the average data range upper limit obtained can be expressed as:

[0201]

[0202] wherein, is the average data range upper limit obtained after the t-1 iteration, is the range upper limit counted for the output data of the operator in the t iteration, is constantly updated with the number of iterations, , is a constant. The input data in the t iteration can be the first training input data or the second training input data.

[0203] Before the iteration, the parameter , , can be set. Generally, the parameter , can be set to 0.

[0204] Any one of the parameters , , can be set randomly or according to a certain rule, or the parameters , , Any of these parameters can be set manually. During the iteration process, the parameters... , , Any one of them can remain unchanged, or it can be adjusted according to certain rules. This application does not impose any restrictions on this.

[0205] Based on the upper limit of the average data range obtained from t iterations, the offset parameter N can be determined:

[0206]

[0207] Where ceil is the floor function. for The number of significant digits is the average number of significant digits after t iterations, and m is the number of digits in the data after reducing the number of digits.

[0208] Taking an operator output format of int32 and a value of m of 16 as an example, .

[0209] For the output of the quantized operator, a saturation operation is performed on values ​​with more than N significant bits, that is, N "1" bits are used to represent values ​​with more than N significant bits. In other words, when the output of the quantized operator is greater than the value represented by N "1" bits, the value is represented by N "1" bits.

[0210] For each value in the output data of the quantized operator, shift it to the right by N bits. Then, a saturation operation can be performed to restrict the result to m bits. It should be understood that m is a preset number. When the result of shifting right by N bits is greater than the range of values ​​that m bits can represent, the result of the saturation operation is taken as all m bits that are "1". When the result of shifting right by N bits is less than or equal to the range of values ​​that m bits can represent, the result of shifting right by N bits is taken as the result of the saturation operation.

[0211] The offset parameter N indicates the P-th bit in the output data of the quantized operator, where P = QN.

[0212] Based on the offset parameter N, only bits P+1 to P+m of the output data of the quantized operator can be retained (or only bits P to P+m-1 can be retained), thereby achieving compression (i.e. format conversion) of the output data of the quantized operator.

[0213] Then, operations can be performed on the first and second training data obtained from the conversion.

[0214] The processing of the input data by the first quantized operator can be expressed as follows:

[0215]

[0216] wherein, is the output of the quantized first operator, is the input data of the first operator quantized by using the data quantization parameter, is the parameter of the first operator quantized by using the operator quantization parameter.

[0217] The processing of the input data by the quantized second operator can be represented as:

[0218]

[0219] wherein is the output of the quantized second operator, is the input data of the second operator quantized by using the operator quantization parameter, is the parameter of the second operator quantized by using the operator quantization parameter.

[0220] Taking the elementwise accumulation operation as an example, the operation result can be represented as:

[0221]

[0222] wherein, R is the operation result, represents converting the data , and the converted result includes the lowest preset number of bits of the data after the data is right shifted by N bits.

[0223] At S850, the data quantization parameter and the operator quantization parameter are adjusted according to the difference between the dequantized training output data and the preset operation result.

[0224] If the training output data is obtained after the conversion of reducing the number of bits, the training output data can be converted reversely, and the data after the reverse conversion is dequantized. That is, the bits on the right side of the converted data can be increased to make the data after the reverse conversion equal to the bits of the output data of the quantized first operator and the second operator. It should be understood that the values of the increased bits can all be “0”. Then, the data after the increase of the bits can be left shifted by N bits to obtain the training output data after the reverse conversion.

[0225] ​For the first operator and the second operator are both conv operators, the training output data obtained by performing a bit-by-bit addition operation on the output of the first operator and the output of the second operator, and the inverse quantization of the training output data can be multiplying the training output data by the product of the scale in the data quantization parameter and the scale in the operator quantization parameter.

[0226] Through S810 to S850, the data quantization parameter and the operator quantization parameter that make the accuracy of the operation result higher can be obtained.

[0227] The parameters of the first operator and the second operator are quantized respectively by using the operator quantization parameter. According to the data quantization parameter, the quantized first operator, the quantized second operator, and the offset parameter N, the data processing system 600 can be determined.

[0228] In the data processing system 600, the operation model 640 can be an operation model in the original neural network model before quantization, or the parameters in the operation model 640 can be obtained by quantizing the parameters in the operation model in the original neural network model.

[0229] In some embodiments, S810 to S850 can be performed by a server. The server can send the data quantization parameter, the quantized first operator, the quantized second operator, the offset parameter, and the like to a terminal device. Thus, the terminal device can determine Figure 9 The data processing system 600 shown.

[0230] Figure 9 is a schematic structural diagram of a data processing system provided by an embodiment of the present application. The data processing system 600 can be located in the computing module 111 of the execution device 110, and the data processing system 600 can be Figure 1 The target model / rule 101 shown. The data processing system 600 can be Figure 1 quantized by the training device 120 or other apparatus after the training of the neural network model is completed.

[0231] The data processing system 600 can also be referred to as a quantized neural network model. The data processing system 600 can be Figure 2 The CNN 200 shown or Figure 3 The CNN 300 shown. Alternatively, each component of the data processing system 600 can be located in one or more CNNs. The data processing system 600 includes a quantized model 610, a first operator 620, a second operator 630, and an operation model 640. The quantized model 610 is configured to quantize first input data of the first operator 620 and second input data of the second operator 630 respectively by using a data quantization parameter.

[0232] The format of the first input data and the format of the second input data can both be floating point numbers. For example, it can be a 32-bit single-precision floating point number (float32), or a 16-bit half-precision floating point number (float16). The quantization model 610 can quantize the first input data and the second input data respectively by using a data quantization parameter to obtain quantized first input data and quantized second input data. The format of the quantized first input data and the format of the quantized second input data can both be 8-bit quantization results (int8).

[0233] The data quantization parameter can include a scale and an offset. The scale is used to represent the increase amount of the floating point number corresponding to the increase of "1" of the quantization result, and the offset is used to represent the ratio of the floating point number represented by the minimum value of the quantization result to the scale. The first operator 620 is used to process the first input data quantized by the quantization model 610 to obtain first operation data. The second operator 630 is used to process the second input data quantized by the quantization model 610 to obtain second operation data. The parameters of the first operator 620 and the parameters of the second operator 630 are quantized by using an operator quantization parameter. That is, when the data processing system 600 is determined, the parameters of the first operator in the neural network model before quantization are quantized by using the operator quantization parameter to obtain the parameters of the first operator 620; the parameters of the second operator in the neural network model before quantization are quantized by using the operator quantization parameter to obtain the parameters of the second operator 620.

[0234] The operator quantization parameter can include a scale and an offset. The determination of the data quantization parameter and the determination of the operator quantization parameter can be referred to the description of Figure 6 to Figure 8 The first operator 620 and the second operator 630 are used to perform the same type of operation. That is, the first operator 620 and the second operator 630 can both be the same type of operator in the neural network model. For example, the first operator 620 and the second operator 630 can both be convolution (conv) operators used for convolution operation, for example, the first operator 620 and the second operator 630 can represent a convolution layer respectively. Each module in the data processing system 600 can be part of the CNN 200 shown in Figure 2 or part of the CNN 300 shown in Figure 3 .

[0235] The first operator 620 and the second operator 630 can also be full connection layers. The excitation function of each neuron of the full connection layer generally adopts a rectified linear unit (ReLU). The output of the first operator 620 and the output of the second operator 630 need to be subsequently operated by the operation model 640. In some embodiments, the first operator 620 and the second operator 630 can be located in different CNNs, and the operation model 640 can be used to process the data output by the different CNNs. Of course, the first operator 620 and the second operator 630 can also be the same type of operators in other types of neural network models.

[0236] When the parameters of the conv operator and the input data of the conv operator are both int8, the output of the conv operator is a quantization result of 32 (int32). That is, when the first operator and the second operator can both be conv operators, and the formats of the parameters of the first operator, the parameters of the second operator, the quantized first input data, and the quantized second input data are all int8, the formats of the output data of the first operator and the second operator are both int32. For the conv operator, the parameters of the conv operator can also be understood as the weights in the conv operator.

[0237] For the conv operator, the processing result of the quantized input data d q1 with the format of int8 with the format of int32 can be represented as:

[0238]

[0239] wherein, is the parameter of the operator obtained by quantizing the operator quantization parameter, and the format is also int8.

[0240] The operation model 640 is used to operate the first operation data and the second operation data. The operation model 640 can perform linear operation on the first operation data and the second operation data. The operation model 640 can also perform bit-by-bit operation on the first operation data and the second operation data, such as bit-by-bit addition or bit-by-bit multiplication operation.

[0241] The data processing system 600 quantizes the input data of the two operators by using the data quantization parameter, and the parameters of the two operators are obtained by using the operator quantization parameter. Then, the output of the two operators can be operated, which avoids the dequantization of the output of the two operators, reduces the calculation amount, reduces the operation power consumption, and improves the data processing performance of the data processing system 600.

[0242] The dequantization operation is a way of vector calculation. In a general NPU, the operation capability of the way of vector calculation is weaker than that of the way of matrix calculation. The way of matrix calculation includes convolution operation and the like. The operation of a neural network model often includes a plurality of serial matrix calculations and a plurality of vector calculations. In general, the computing power of a processor for matrix calculation is higher than that for vector calculation. When a neural network model needs to perform a large amount of vector calculation, the matrix calculation depending on the result of the vector calculation is in a waiting state in the case where the vector calculation is not completed, resulting in a flow interruption and a performance bottleneck (referred to as vector bound).

[0243] By quantizing the first operator and the second operator by using the same quantization parameter and quantizing the input data of the first operator and the input data of the second operator by using the same parameter, the operation module 640 of the data processing system 600 can perform operation on the output of the first operator and the output of the second operator, and the data processing system 600 can reduce the dequantization operation required by the neural network model, thereby relieving the vector bound and effectively improving the data processing capability of the neural network model.

[0244] The data processing system 600 can further include a format conversion model 650. The format conversion model 650 can be used for data compression and can also be referred to as a compression model. The format conversion model 650 is used to reduce the number of bits of the first original operation data output by the first operator 620 to obtain the first operation data. The format conversion model 650 is also used to reduce the number of bits of the second original operation data output by the second operator 630 to obtain the second operation data.

[0245] For example, the format conversion model 650 is used to convert the formats of the first original operation data and the second original operation data in int32 into int16, respectively. The int16 data obtained by format conversion of the first original operation data output by the first operator 620 is the first operation data, and the int16 data obtained by format conversion of the second original operation data output by the second operator 630 is the second operation data. The format conversion model 650 can determine the first operation data and the second operation data according to the offset parameter. The processing process of the format conversion model 650 can be understood as compression of data.

[0246] As shown in (A) of FIG. 13, Figure 10 As shown in (A) of FIG. 13,

[0247] As shown in (A) of FIG. 13,Figure 10 When the bit positions before the bit position indicated by the offset parameter in the first original operation data are not all 0, and there is a bit position with a value of 1, the first operation data includes a preset number of bit positions that are all "1".

[0248] Similarly, when the bit positions before the bit position indicated by the offset parameter in the second original operation data are all 0, the second operation data includes the bit position indicated by the offset parameter in the second original operation data and a total of a preset number of bit positions after the bit position.

[0249] When the bit positions before the bit position indicated by the offset parameter in the second original operation data are not all 0, and there is a bit position with a value of 1, the second operation data includes a preset number of bit positions that are all "1". The format conversion model 650 can perform right shift operation and saturation operation on any one of the first original operation data or the second original operation data.

[0250] The right shift operation can be represented as:

[0251]

[0252] wherein "»" is a right shift symbol, conv out represents an original operation result, conv out represents a right shift operation result, and N is a number of bits to be right shifted. It should be understood that the number of bits N to be right shifted is less than or equal to a sum of bit positions of the data output by the format conversion model 650. conv out and the number of bits in the original operation result.

[0253] The right shift operation result can be subjected to saturation operation:

[0254]

[0255] wherein, conv INT16 represents a saturation operation result, and p is a difference between the number of bits in the original operation result conv out and the number of bits N to be right shifted. The clip(a, b, c) operator represents limiting a to be between b and c, when a is less than b, the operation result is b; when a is greater than or equal to b and a is less than or equal to c, the operation result is a; when a is greater than or equal to c, the operation result is c.

[0256] When the format of the original operation data is int32, that is, the original operation data includes 32 bits, and the bit number of the operation result, that is, the preset number, is 16, p = 32-N, and N≤16.

[0257] The bit number m of the operation result of the operator clip can be the same as the bit number of the original operation result, and the lowest preset number m of bits in the saturated operation result can be taken as the operation result. That is, the lowest m bits in the saturated operation result are the operation data corresponding to the original operation data.

[0258] That is, the original operation data can be right shifted by N bits. The size of the result of right shifting the binary number with the same bit number as the original operation result and each bit being 1 by N bits and the right shift result of the original operation data is determined. When the right shift result of the original operation data is larger, the lowest preset number of bits in the right shift result of the original operation data is taken as the operation data; otherwise, the preset number of "1"s is taken as the operation result.

[0259] The first operation data can be the first original operation data, or data obtained by right shift operation and saturation operation on the first original operation data. The second operation data can be the second original operation data, or data obtained by right shift operation and saturation operation on the second original operation data.

[0260] After the right shift operation and the saturation operation, the format conversion model 650 can convert the original operation data into operation data, and take the operation data as the input of the operation module 640.

[0261] Through the conversion of the data format by the format conversion model 650, the calculation amount of the operation module 640 can be reduced, and thus the data processing performance of the data processing system 600 can be improved.

[0262] The quantized neural network model can be used to implement Figure 11 or Figure 12 the data processing method. Figure 11 is a schematic flowchart of a data processing method provided by an embodiment of the present application. The data processing method can be executed by the computing module 111 in the execution device 110 shown in Figure 1 .

[0263] S1201, acquire a quantized neural network model, the quantized neural network model being obtained by quantizing an original neural network model, the original neural network model comprising a first operator, a second operator and a first operation module, the first operator being used for the same type of operation as the second operator, and the first operation module being used for performing a first operation on an output of the first operator and an output of the second operator.

[0264] S1202, process first input data of the third operator and second input data of the fourth operator by using the quantized neural network model, the quantized neural network model comprising a quantization module, a first operator, a second operator and a first operation module, the quantization module being used for quantizing the first input data and the second input data by using a data quantization parameter, the second operation module being used for performing the first operation, the third operator being a quantized first operator, the fourth operator being a quantized second operator, and the data quantization parameter being determined according to a range of first training input data of the first operator and a range of second training input data of the second operator.

[0265] The second operation module can be used for performing the first operation on first operation data and second operation data. The first operation data is obtained by performing the first operation on the quantized first input data by using the third operator, and the second operation data is obtained by performing the second operation on the quantized second input data by using the fourth operator. The quantized neural network model obtained by quantizing the original neural network model performs the same operation as the original neural network model, and only the accuracy of the operation result changes.

[0266] The data quantization parameter is determined according to the range of the first training input data of the first operator and the range of the second training input data of the second operator, thereby improving the data processing accuracy of the quantized neural network model.

[0267] By using the data quantization parameter to process the first input data and the second input data, the second operation module can perform the operation on the first operation data and the second operation data without performing the operation on the first operation data and the second operation data after the dequantization. The first operation data is obtained by processing the quantized first input data by using the third operator, and the second operation data is obtained by processing the quantized second input data by using the fourth operator.

[0268] Through S1201 to S1202, the operation accuracy of the quantized neural network model is improved, the demand of the quantized neural network model for the dequantization operation is reduced, the operation resources are saved, and the processing efficiency is improved.

[0269] Optionally, the data quantization parameter is obtained by adjusting an initial data quantization parameter, and the adjusting is to minimize the difference between the actual training output data and the preset training output data.

[0270] The initial quantization parameter is determined according to a range of the first training input data and a range of the second training input data.

[0271] The preset training output data corresponds to a training input data group, and the training input data group includes the first training input data and the second training input data.

[0272] The actual training output data is obtained by processing the first training input data and the second training input data by using the quantized neural network model, and the quantization module is configured to quantize the first training input data and the second training input data by using the initial data quantization parameter.

[0273] An initial quantization parameter is determined according to a range of the first training input data and a range of the second training input data. The first training input data and the second training input data are processed by using a quantized neural network model to obtain actual training output data, wherein a quantization module uses the initial data quantization parameter to quantize the first training input data and the second training input data. The initial data quantization parameter is adjusted to minimize the difference between the actual training output data and preset training output data, thereby obtaining a data quantization parameter.

[0274] It should be understood that in the quantized neural network model, a third operator is configured to perform a first operation on the quantized first training input data to obtain first training operation data; a fourth operator is configured to perform a second operation on the quantized second training input data to obtain second training operation data; and a second operation module is configured to perform a third operation on the first training operation data and the second training operation data to obtain the actual training output data.

[0275] Since the data quantization parameter minimizes the difference between the actual training output data and the preset training output data, the quantized neural network model has higher accuracy.

[0276] Minimizing the difference between the actual training output data and the preset training output data can be understood as gradually adjusting the initial data quantization parameter of the initial action recognition system according to the difference between the actual training output data and the preset training output data until the difference between the actual training output data and the preset training output data is within a certain preset range, or when the number of adjustments reaches a preset number, the initial data quantization parameter at this time is determined as the adjusted data quantization parameter.

[0277] Optionally, the parameter of the third operator is obtained by quantizing the parameter of the first operator using an operator quantization parameter, the parameter of the fourth operator is obtained by quantizing the parameter of the second operator using the operator quantization parameter, and the operator quantization parameter is determined according to a parameter range of the first operator and a parameter range of the second operator.

[0278] The operator quantization parameter is determined according to the parameter range of the first operator and the parameter range of the second operator, and the parameter of the first operator and the parameter of the second operator are quantized using the operator quantization parameter to obtain the parameter of the third operator and the parameter of the fourth operator. In the case of quantization to reduce the operation amount, the data processing accuracy of the quantized neural network model is improved.

[0279] Optionally, the quantized neural network model further comprises a compression module configured to compress the output of the third operator and the output of the fourth operator according to an offset parameter to obtain the first operation data and the second operation data, the offset parameter is used to indicate the position of the highest bit in the compressed data before the compression, and the second operation module is configured to perform the first operation on the compressed data.

[0280] The offset parameter is determined according to the number of significant bits of the first training operation data and the number of significant bits of the second training operation data, the first training operation data is obtained by processing the first training input data quantized using the data quantization parameter using the third operator, and the second training operation data is obtained by processing the second training input data quantized using the data quantization parameter using the fourth operator.

[0281] According to the number of significant bits of the first training operation data and the number of significant bits of the second training operation data, the data processing accuracy of the quantized neural network model is improved while reducing the operation amount.

[0282] Figure 12 is a schematic flowchart of a data processing method provided by an embodiment of the present application.

[0283] The data processing method 700 comprises S710 to S720. The data processing method 700 can be executed in the computing module 111 of the execution device 110 shown in Figure 1

[0284] In S710, the first input data of a first operator in a neural network model and the second input data of a second operator in the neural network model are quantized using a data quantization parameter.

[0285] ​In S720, the first processing information and the second processing information are processed. The first processing information is obtained by processing the quantized first input data using the first operator, and the second processing information is obtained by processing the quantized second input data using the second operator.

[0286] Through steps S710 and S720, the first input data of the first operator and the second input data of the second operator are quantized using the same data quantization parameters, so that the outputs of the first operator and the second operator can be directly calculated without dequantization or other processing, thereby improving the data processing efficiency of the neural network model.

[0287] The first parameter of the first operator and the second parameter of the second operator can be floating-point numbers, or they can be obtained by quantizing the floating-point number parameters using the operator quantization parameters.

[0288] The first and second parameters are obtained through quantization, which reduces the size of the first and second operators and decreases their resource consumption during data processing. Furthermore, the quantization of both the first and second parameters utilizes operator quantization, allowing the data processing results of the first and second operators to be directly calculated without the need for inverse quantization or other processing, thus improving the data processing efficiency of the neural network model.

[0289] To improve the computational accuracy of the first and second operators after quantization, the operator quantization parameters can be obtained based on the ranges of the first and second parameters.

[0290] The operator quantization parameters can be determined by... Figure 1 The training device 120 or other device shown is used for execution. Of course, the device that determines the data quantization parameters can be the same as or different from the device that executes S710 to S720.

[0291] The operator quantization parameters can be obtained based on the maximum and minimum values ​​of the first and second parameters. For example, the operator quantization parameters may include scale and offset. The scale of the operator quantization parameter can be obtained by equally dividing the difference between the maximum and minimum values ​​of the first and second parameters according to the number of bits in the quantization result. The offset of the operator quantization parameter can be determined based on the ratio of the minimum value of the first and second parameters to the scale of the operator quantization parameter.

[0292] To improve the computational accuracy of the first and second operators after quantization, the data quantization parameters can be determined based on the range of data processed by the first and second operators.

[0293] The determination of data quantization parameters can be made by Figure 1The training device 120 or other devices shown performs. Of course, the device for determining the data quantization parameter and the device for performing S710 to S720 can be the same or different devices.

[0294] Specifically, a training data set can be obtained, the training data set comprising first training input data, second training input data. The first training input data is the input data of the first operator before quantization, and the second training input data is the input data of the second operator before quantization. The data quantization parameter can be determined according to the range of the first training input data and the range of the second training input data.

[0295] For example, the data quantization parameter can be determined by a plurality of first training input data and a plurality of second training input data. Each first training input data includes a plurality of numerical values, and each second training input data includes a plurality of numerical values. The average maximum value in each first training input data and each second training input data can be used as the maximum value that the quantization result of the data quantization parameter can represent, and the average minimum value in each first training input data and each second training input data can be used as the minimum value that the quantization result of the data quantization parameter can represent. The average maximum value can be a weighted average of a plurality of maximum values, and the average minimum value can be a weighted average of a plurality of minimum values. The weight can be understood as the degree of influence of the maximum value or the minimum value in each training input data in the first training input data, the second training input data on the data quantization parameter. Specifically, please refer to the description of Figure 8 .

[0296] In order to improve the calculation accuracy of the first operator and the second operator after quantization, before the data quantization parameter and the operator quantization parameter are used for the processing of actual data, the difference between the result after the processing result of the first training input data after quantization by the first operator and the processing result of the second training input data by the second operator is operated, and the preset operation result, the data quantization parameter and / or the operator quantization parameter can be adjusted.

[0297] Specifically, the training data set further comprises a preset operation result corresponding to the first training input data and the second training input data. The preset operation result corresponding to the first training input data and the second training input data can be the operation result obtained by operating the processing result of the first training input data by the first operator before quantization and the processing result of the second training input data by the second operator before quantization. Alternatively, the preset operation result can be artificially set. The format of the preset operation result can be a floating point number.

[0298] The operation in S720 can be performed on the first training operation data and the second training operation data to obtain training output data. The first training operation data is obtained by processing the first training input data quantized by using the data quantization parameter by the first operator, and the second training operation data is obtained by processing the second training input data quantized by using the data quantization parameter by the second operator.

[0299] The training output data can be dequantized. The data quantization parameter and / or the operator quantization parameter can be adjusted according to the difference between the dequantization result of the training output data and the preset operation result.

[0300] To further reduce the operation amount of the neural network model, the operation result of the first operator and the operation result of the second operator can be processed to reduce the number of bits.

[0301] The first operator processes the quantized first input data and outputs first original operation data. The second operator processes the quantized second input data and outputs second original operation data.

[0302] The preset number of bits with the highest number of bits in the first original operation data can be taken as the first operation result, and the preset number of bits with the highest number of bits in the second original operation data can be taken as the second operation data for subsequent operation. The preset number of bits with the highest number of bits is the preset number of bits at the left end.

[0303] Alternatively, the first operation data and / or the second operation data can be determined according to the offset parameter.

[0304] When the bits before the bit indicated by the offset parameter in the first original operation data are all 0, the first operation data includes the preset number of bits after the bit indicated by the offset parameter in the first original operation data.

[0305] When the bits before the bit indicated by the offset parameter in the first original operation data are not all 0, there is a bit with a value of 1, and the first operation data includes the preset number of bits, all of which are “1”.

[0306] Similarly, when the bits before the bit indicated by the offset parameter in the second original operation data are all 0, the second operation data includes the preset number of bits after the bit indicated by the offset parameter in the second original operation data.

[0307] When the bits before the bit indicated by the offset parameter in the second original operation data are not all 0, there is a bit with a value of 1, and the second operation data includes the preset number of bits, all of which are “1”.

[0308] Optionally, the first original operation data and the second original operation data are compressed by reducing the number of bits.

[0309] It should be understood that if the processing result of the first operator or the processing result of the second operator has valid data at the bit indicated by the offset parameter or higher, the first operation data corresponding to the processing result of the first operator or the second operation data corresponding to the processing result of the second operator can be represented as a preset number of “1”. This mode can also be understood as a saturation operation. That is, when the processing result is greater than the maximum value that can be represented by the number of bits after the bit indicated by the offset parameter, the processing result is represented as all “1” in the preset number of bits, that is, the maximum value that can be represented by the preset number of bits.

[0310] The offset parameter can be obtained according to the processing result of the first operator on the first training input data and the processing result of the second operator on the second training input data.

[0311] The offset parameter can be determined according to the number of valid bits of the data output by the first operator processing the quantized first training input data and the number of valid bits of the data output by the second operator processing the quantized second training input data.

[0312] For example, the first operator can process a plurality of quantized first training parameters, and the processing result of the first operator on each quantized first training parameter includes a plurality of numbers. The plurality of numbers can form a matrix or a vector, etc. The second operator can process a plurality of quantized second training parameters, and the processing result of the second operator on each quantized second training parameter includes a plurality of numbers. The offset parameter can be determined according to the average of the maximum number of valid bits in each processing result. For example, the average can be rounded up, and the offset parameter is used to indicate the highest bit of the number of valid bits obtained by rounding up the average.

[0313] According to the difference between the dequantization result of the training output data and the preset operation result, the offset parameter can also be adjusted. Thus, the accuracy and precision of the data processing result are higher.

[0314] In the process of processing image, audio and other data by the neural network model, a plurality of operators are generally used. Before performing the neural network model quantization method provided in the embodiments of the present application, the original neural network model can be traversed to determine the processing structure of the original neural network model including the first operator, the second operator and the operation model for operating the output of the first operator and the output of the second operator. Figure 9 Taking the first operator and the second operator as convolution operators and the operation model as an eltwise operator as an example for description.

[0315] Figure 13 is a schematic flowchart of a processing structure identification method provided by an embodiment of the present application. The processing structure identification method can be executed by the training device 120 or other devices shown in Figure 1 .

[0316] At S910, it is determined whether the node corresponding to the node i is a convolution operator. If the node i is not a convolution operator, i is set to i+1, and S910 is performed again. If the node i is a convolution operator, S920 is performed.

[0317] At S920, it is determined whether the output data of the node i is an input of an eltwise operator. If the output data of the node i is not an input of an eltwise operator, i is set to i+1, and S910 is performed again. If the output data of the node i is an input of an eltwise operator, S930 is performed.

[0318] At S930, it is determined whether the other input of the eltwise operator is output data of a convolution operator. If the other input of the eltwise operator is not output data of a convolution operator, i is set to i+1, and S910 is performed again. If the other input of the eltwise operator is output data of a convolution operator, the node i is taken as a first operator, and a convolution operator providing the other input of the eltwise operator is taken as a second operator, and the method 800 is performed. In addition, i is set to i+1, and S910 is performed again. When i is greater than the amount of node data in the model in the neural network, the S910 is stopped. Through the method 900, the structure that all two-way convolution operators in the neural network model output results to an eltwise operator can be determined.

[0319] The data processing system, the neural network model quantization method, and the data processing method provided by the embodiments of the present application are described above in combination with Figure 1 to Figure 13 . The device embodiment of the embodiments of the present application is described below in combination with Figure 14 to Figure 17 . It should be understood that the description of the data processing system, the neural network model quantization method, and the data processing method corresponds to the description of the device embodiment, and thus, the parts not described in detail can be referred to the description above.

[0320] Figure 14 is a schematic structural diagram of a neural network model quantization device provided by an embodiment of the present application. The neural network model quantization device 3000 can be located in the training device 120 or other devices shown in Figure 1 . The neural network model quantization device 3000 includes a storage module 3010 and a processing module 3020. The storage module 3010 is configured to store a program.

[0321] When the program runs in the processing module 3020, the processing module 3020 is configured to: acquire an original neural network model, the original neural network model comprising a first operator, a second operator, and a first operation module, the first operator being configured to perform a first operation, the second operator being configured to perform a second operation, the first operation and the second operation being operations of the same type, and the first operation module being configured to perform a third operation on an output of the first operator and an output of the second operator; determine a data quantization parameter according to a range of first training input data and a range of second training input data, the first training input data being input data of the first operator, and the second training input data being input data of the second operator; and determine a quantized neural network model according to the original neural network model, the quantized neural network model comprising a quantization module, a third operator, a fourth operator, and a second operation module, the quantization module being configured to quantize first input data of the third operator and second input data of the fourth operator using the data quantization parameter respectively, the third operator being a quantized first operator, the fourth operator being a quantized second operator, and the second operation module being configured to perform the first operation.

[0322] Optionally, the processing module 3020 is further configured to acquire preset training output data corresponding to a training input data set, the training input data set comprising the first training input data and the second training input data.

[0323] The processing module 3020 is further configured to quantize the first training input data and the second training input data using the data quantization parameter respectively. The processing module 3020 is further configured to process the quantized first training input data and the quantized second training input data using the quantized neural network model to obtain actual training output data. The processing module 3020 is further configured to adjust the data quantization parameter according to a difference between the actual training output data and the preset training output data, so as to minimize the difference.

[0324] The quantization module is configured to quantize the first input data of the third operator and the second input data of the fourth operator using the adjusted data quantization parameter respectively. Optionally, the processing module 3020 is further configured to determine an operator quantization parameter according to a parameter range of the first operator and a parameter range of the second operator.

[0325] The processing module 3020 is further configured to quantize a parameter of the first operator using the operator quantization parameter to obtain a parameter of the third operator. The processing module 3020 is further configured to quantize a parameter of the second operator using the operator quantization parameter to obtain a parameter of the fourth operator.

[0326] Optionally, the quantized neural network model further comprises a compression module, configured to compress outputs of the third operator and the fourth operator respectively according to an offset parameter, the offset parameter being used to indicate a position of a highest bit in the compressed data before the compression, and the second operation module is configured to perform the first operation on the compressed data.

[0327] The processing module 3020 is further configured to quantize the first training input data and the second training input data respectively by using the data quantization parameter. The processing module 3020 is further configured to process the quantized first training input data by using the third operator, and the third operator outputs first training operation data. The processing module 3020 is further configured to process the quantized second training input data by using the fourth operator, and the fourth operator outputs second training operation data. The processing module 3020 is further configured to determine the offset parameter according to the number of significant bits of the first training operation data and the number of significant bits of the second training operation data.

[0328] Figure 15 is a schematic structural diagram of a data processing apparatus provided by an embodiment of the present application. The data processing apparatus 2000 can be located in the execution device 110 or other devices shown in the figure. The data processing apparatus 2000 comprises a storage module 2010 and a processing module 2020. The storage module 2010 is configured to store a program. Figure 1

[0329] When the program runs in the processing module 2020, the processing module 2020 is configured to: obtain a quantized neural network model, the quantized neural network model being obtained by quantizing an original neural network model, the original neural network model comprising a first operator, a second operator and a first operation module, the first operator and the second operator being used to perform the same type of operation, and the first operation module being used to perform a first operation on an output of the first operator and an output of the second operator; process first input data of the third operator and second input data of the fourth operator by using the quantized neural network model, the quantized neural network model comprising a quantization module, a first operator, a second operator and a second operation module, the quantization module being configured to quantize the first input data and the second input data respectively by using a data quantization parameter, the second operation module being configured to perform the first operation, the third operator being a quantized first operator, the fourth operator being a quantized second operator, and the data quantization parameter being determined according to a range of first training input data of the first operator and a range of second training input data of the second operator.

[0330] ​Optionally, the data quantization parameter is obtained by adjusting an initial data quantization parameter, and the adjusting is to minimize a difference between the actual training output data and the preset training output data.

[0331] The initial quantization parameter is determined according to a range of the first training input data and a range of the second training input data. The preset training output data corresponds to a training input data group, and the training input data group includes the first training input data and the second training input data.

[0332] The actual training output data is obtained by processing the first training input data and the second training input data by using the quantized neural network model. The quantization module is configured to quantize the first training input data and the second training input data by using the initial data quantization parameter, respectively.

[0333] Optionally, the parameter of the third operator is obtained by quantizing the parameter of the first operator by using an operator quantization parameter, and the parameter of the fourth operator is obtained by quantizing the parameter of the second operator by using the operator quantization parameter. The operator quantization parameter is determined according to a parameter range of the first operator and a parameter range of the second operator.

[0334] Optionally, the quantized neural network model further includes a compression module configured to compress an output of the third operator and an output of the fourth operator by using an offset parameter. The offset parameter is used to indicate a position of a highest bit in data after the compression in data before the compression. The second operation module is configured to perform the first operation on the compressed data.

[0335] The offset parameter is determined according to a number of effective bits of first training operation data and a number of effective bits of second training operation data. The first training operation data is obtained by processing the first training input data quantized by using the data quantization parameter by using the third operator. The second training operation data is obtained by processing the second training input data quantized by using the data quantization parameter by using the fourth operator.

[0336] Figure 16 FIG. 1 is a hardware structure schematic diagram of a data processing apparatus according to an embodiment of the present application. Figure 16 The data processing apparatus 4000 shown in the figure includes a memory 4001, a processor 4002, a communication interface 4003, and a bus 4004. The memory 4001, the processor 4002, and the communication interface 4003 are communicatively connected to each other through the bus 4004.

[0337] The memory 4001 can be a ROM, a static storage device, and a RAM. The memory 4001 can store programs, and when the programs stored in the memory 4001 are executed by the processor 4002, the processor 4002 and the communication interface 4003 are used to perform various steps of the data processing method of the embodiments of the present application.

[0338] The processor 4002 can be a general-purpose CPU, a microprocessor, an ASIC, a GPU, or one or more integrated circuits, used to execute related programs to realize the functions required to be performed by the units in the data processing device of the embodiments of the present application, or to execute the data processing method of the method embodiments of the present application.

[0339] The processor 4002 can also be an integrated circuit chip with signal processing capability, for example, it can be a Figure 4 chip as shown. In the implementation process, various steps of the data processing method of the embodiments of the present application can be completed by the integrated logic circuit of hardware in the processor 4002 or the instructions in the form of software.

[0340] The above-mentioned processor 4002 can also be a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware coding processor for execution, or a combination of hardware and software modules in the coding processor. The software modules can be located in the random access memory, the flash memory, the read-only memory, the programmable read-only memory, the electrically erasable programmable memory, the register, or other mature storage mediums in the art. The storage medium is located in the memory 4001, and the processor 4002 reads the information in the memory 4001, and combines the hardware to complete the functions required to be performed by the units included in the data processing device of the embodiments of the present application, or executes the data processing method of the method embodiments of the present application.

[0341] The communication interface 4003 uses a transceiver such as but not limited to a transceiver to realize the communication between the device 4000 and other devices or communication networks. For example, the to-be-processed image can be obtained through the communication interface 4003.

[0342] The bus 4004 can include a path for transmitting information between various components (for example, the memory 4001, the processor 4002, the communication interface 4003) of the device 4000.

[0343] Figure 17 is a hardware structure schematic diagram of the neural network model quantization device of the embodiments of the present application. Similar to the above-mentioned device 4000,Figure 17 The neural network model quantization apparatus 5000 shown includes a memory 5001, a processor 5002, a communication interface 5003, and a bus 5004. The memory 5001, the processor 5002, and the communication interface 5003 are in communication connection with each other through the bus 5004.

[0344] The neural network model quantization apparatus 5000 shown can quantize the original neural network model, and the quantized neural network model can be used to perform the data processing method of the embodiments of the present application. Figure 17 The apparatus shown can obtain the training data set and the original neural network model required for quantization from the outside through the communication interface 5003, and then the processor can quantize the neural network model according to the training data set and the original neural network model.

[0345] Specifically, Figure 17 The apparatus shown can obtain the training data set and the original neural network model required for quantization from the outside through the communication interface 5003, and then the processor can quantize the neural network model according to the training data set and the original neural network model.

[0346] It should be noted that although the apparatus 4000 and the apparatus 5000 described above only show the memory, the processor, and the communication interface, in the specific implementation process, those skilled in the art should understand that the apparatus 4000 and the apparatus 5000 can also include other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the apparatus 4000 and the apparatus 5000 can also include hardware devices for realizing other additional functions. In addition, those skilled in the art should understand that the apparatus 4000 and the apparatus 5000 can also only include devices necessary for realizing the embodiments of the present application, and do not necessarily include all the devices shown in the apparatus 4000 and the apparatus 5000. Figure 16 and Figure 17 the apparatus 4000 and the apparatus 5000.

[0347] It should be understood that the processor in the embodiments of the present application can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0348] It should also be understood that the memory in the embodiments of the present application can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example, and not limitation, many forms of random access memory (RAM) can be used, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0349] The above-described embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the above-described embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server, or data center to another via wired (for example, infrared, wireless, microwave, etc.) or wireless means. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0350] It should be understood that the term "and / or" herein merely describes an association relationship of associated objects, which means that there can be three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone, where A and B can be singular or plural. In addition, the character " / " herein generally represents an "or" relationship between the associated objects before and after it, but it can also represent an "and / or" relationship. The specific meaning can be understood according to the context before and after it.

[0351] In this application, "at least one" means one or more, and "multiple" means two or more. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0352] It should be understood that in various embodiments of the present application, the size of the sequence number of the above-described processes does not mean the order of execution, and the execution order of the processes should be determined according to their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0353] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0354] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0355] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are merely schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0356] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0357] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit.

[0358] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0359] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A neural network model quantization method, characterized in that, The method comprises: obtaining an original neural network model, the original neural network model being applied to multiple fields including images, videos or voices, comprising a first operator, a second operator and a first operation module, the first operator and the second operator being used for performing the same type of operation, and the first operation module being used for performing a first operation on the output of the first operator and the output of the second operator; determining a data quantization parameter according to the range of first training input data and the range of second training input data, the first training input data being the input data of the first operator, and the second training input data being the input data of the second operator; determining a quantized neural network model according to the original neural network model, the quantized neural network model being deployed on a computing device and being implemented by a processor of the computing device, comprising a quantization module, a third operator, a fourth operator and a second operation module, the quantization module being used for quantizing the first input data of the third operator and the second input data of the fourth operator respectively by using the data quantization parameter, the third operator being a quantized first operator, the fourth operator being a quantized second operator, and the second operation module being used for performing the first operation on the output of the third operator and the output of the fourth operator.

2. The method of claim 1, wherein, The method further comprises: obtaining preset training output data corresponding to a training input data set, the training input data set comprising the first training input data and the second training input data; quantizing the first training input data and the second training input data respectively by using the data quantization parameter; processing the quantized first training input data and the quantized second training input data by using the quantized neural network model to obtain actual training output data; adjusting the data quantization parameter according to the difference between the actual training output data and the preset training output data to minimize the difference; the quantization module is used for quantizing the first input data of the third operator and the second input data of the fourth operator respectively by using the adjusted data quantization parameter.

3. The method of claim 2, wherein, The method further comprises: determining an operator quantization parameter according to the parameter range of the first operator and the parameter range of the second operator; quantizing the parameters of the first operator by using the operator quantization parameter to obtain the parameters of the third operator; quantizing the parameters of the second operator by using the operator quantization parameter to obtain the parameters of the fourth operator.

4. The method according to any one of claims 1 to 3, characterized in that, The quantized neural network model further comprises a compression module, the compression module being used for compressing the output of the third operator and the output of the fourth operator respectively according to an offset parameter, the offset parameter being used for indicating the position of the highest bit in the data after the compression in the data before the compression, and the second operation module being used for performing the first operation on the compressed data; The method further comprises: quantizing the first training input data and the second training input data respectively by using the data quantization parameter; processing the quantized first training input data by using the third operator, the third operator outputting first training operation data; processing the quantized second training input data by using the fourth operator, the fourth operator outputting second training operation data; determining the offset parameter according to the number of significant bits of the first training operation data and the number of significant bits of the second training operation data.

5. A data processing method, characterized by, The method comprises: obtaining a quantized neural network model, the quantized neural network model being obtained by quantizing an original neural network model, the original neural network model being applied to multiple fields, the multiple fields including images, videos, or voices, the quantized neural network model comprising a first operator, a second operator, and a first operation module, the first operator and the second operator being used for performing the same type of operation, the first operation module being used for performing a first operation on an output of the first operator and an output of the second operator, the quantized neural network model being deployed on a computing device and being implemented by a processor of the computing device, the quantized neural network model comprising a quantization module, a third operator, a fourth operator, and a second operation module, the second operation module being used for performing the first operation on an output of the third operator and an output of the fourth operator, the third operator being the quantized first operator, the fourth operator being the quantized second operator; obtaining first input data and second input data; instructing the quantization module to quantize the first input data and the second input data according to a data quantization parameter, the data quantization parameter being determined according to a range of first training input data of the first operator and a range of second training input data of the second operator; inputting the quantized first input data into the third operator and inputting the quantized second input data into the fourth operator to obtain first operation data output by the third operator and second operation data output by the fourth operator; instructing the second operation module to perform the first operation on the first operation data and the second operation data to obtain output data.

6. The method of claim 5, wherein the data quantization parameter is obtained by adjusting an initial data quantization parameter, the adjustment minimizing a difference between actual training output data and preset training output data, the initial data quantization parameter is determined according to the range of the first training input data and the range of the second training input data, the preset training output data corresponds to a training input data group, the training input data group comprising the first training input data and the second training input data, the actual training output data is obtained by processing the first training input data and the second training input data by using the quantized neural network model, the quantization module being used for quantizing the first training input data and the second training input data by using the initial data quantization parameter.

7. The method of claim 6, wherein, The parameter of the third operator is obtained by quantizing the parameter of the first operator by using an operator quantization parameter, the parameter of the fourth operator is obtained by quantizing the parameter of the second operator by using the operator quantization parameter, and the operator quantization parameter is determined according to a parameter range of the first operator and a parameter range of the second operator.

8. The method according to any one of claims 5-7, characterized in that, The quantized neural network model further comprises a compression module configured to compress the output of the third operator and the output of the fourth operator respectively according to an offset parameter, the offset parameter being used to indicate the position of the highest bit in the data after the compression in the data before the compression, and the second operation module being configured to perform the first operation on the compressed data. The offset parameter is determined according to the number of effective bits of the first training operation data and the number of effective bits of the second training operation data, the first training operation data being obtained by processing the first training input data quantized by using the data quantization parameter by using the third operator, and the second training operation data being obtained by processing the second training input data quantized by using the data quantization parameter by using the fourth operator.

9. A neural network model quantization apparatus, comprising: The device comprises a storage module and a processing module, The storage module is configured to store a program; When the program runs in the processing module, the processing module is configured to: obtain an original neural network model, the original neural network model being applied to multiple fields, the multiple fields comprising images, videos or speech, the original neural network model comprising a first operator, a second operator and a first operation module, the first operator and the second operator being configured to perform the same type of operation, and the first operation module being configured to perform a first operation on the output of the first operator and the output of the second operator; determine a data quantization parameter according to the range of the first training input data and the range of the second training input data, the first training input data being the input data of the first operator, and the second training input data being the input data of the second operator; determine a quantized neural network model according to the original neural network model, the quantized neural network model being deployed on a computing device and implemented by a processor of the computing device, and comprising a quantization module, a third operator, a fourth operator and a second operation module, the quantization module being configured to quantize the first input data of the third operator and the second input data of the fourth operator by using the data quantization parameter, the third operator being a quantized first operator, the fourth operator being a quantized second operator, and the second operation module being configured to perform the first operation on the output of the third operator and the output of the fourth operator.

10. The device of claim 9, wherein the processing module is further configured to obtain preset training output data corresponding to a training input data set, the training input data set comprising the first training input data and the second training input data; the processing module is further configured to quantize the first training input data and the second training input data by using the data quantization parameter respectively. The processing module is further configured to process the quantized first training input data and the quantized second training input data by using the quantized neural network model to obtain actual training output data. The processing module is further configured to adjust the data quantization parameter according to a difference between the actual training output data and the preset training output data, so as to minimize the difference. The quantization module is configured to quantize the first input data of the third operator and the second input data of the fourth operator by using the adjusted data quantization parameter.

11. The apparatus of claim 10, wherein The processing module is further configured to determine an operator quantization parameter according to a parameter range of the first operator and a parameter range of the second operator. The processing module is further configured to quantize the parameters of the first operator by using the operator quantization parameter to obtain the parameters of the third operator. The processing module is further configured to quantize the parameters of the second operator by using the operator quantization parameter to obtain the parameters of the fourth operator.

12. The apparatus of any one of claims 9-11, wherein, The quantized neural network model further comprises a compression module configured to compress the output of the third operator and the output of the fourth operator according to an offset parameter, the offset parameter being used to indicate a position of a highest bit in the compressed data in the data before the compression, and the second operation module being configured to perform the first operation on the compressed data. The processing module is further configured to quantize the first training input data and the second training input data by using the data quantization parameter. The processing module is further configured to process the quantized first training input data by using the third operator, the third operator outputting first training operation data. The processing module is further configured to process the quantized second training input data by using the fourth operator, the fourth operator outputting second training operation data. The processing module is further configured to determine the offset parameter according to a number of significant bits of the first training operation data and a number of significant bits of the second training operation data.

13. A data processing apparatus, characterized by: comprising: a storage module and a processing module, The storage module is configured to store a program. When the program runs in the processing module, the processing module is configured to: obtain a quantized neural network model, the quantized neural network model being obtained by quantizing an original neural network model, the original neural network model being applied to multiple fields, the multiple fields including images, videos, or voices, the original neural network model comprising a first operator, a second operator, and a first operation module, the first operator and the second operator being configured to perform the same type of operation, and the first operation module being configured to perform a first operation on an output of the first operator and an output of the second operator, The quantized neural network model is deployed on a computing device and implemented by a processor of the computing device, and includes a quantization module, a third operator, a fourth operator, and a second operation module configured to perform the first operation on an output of the third operator and an output of the fourth operator, the third operator being the quantized first operator, and the fourth operator being the quantized second operator. obtaining first input data and second input data; indicating the quantization module to quantize the first input data and the second input data according to a data quantization parameter, the data quantization parameter being determined according to a range of first training input data of the first operator and a range of second training input data of the second operator; inputting the quantized first input data into the third operator and inputting the quantized second input data into the fourth operator to obtain first operation data output by the third operator and second operation data output by the fourth operator; indicating the second operation module to perform the first operation on the first operation data and the second operation data to obtain output data.

14. The apparatus of claim 13, wherein the data quantization parameter is obtained by adjusting an initial data quantization parameter, the adjustment minimizing a difference between actual training output data and preset training output data, the initial data quantization parameter is determined according to the range of the first training input data and the range of the second training input data, the preset training output data corresponds to a training input data set, the training input data set including the first training input data and the second training input data, the actual training output data is obtained by processing the first training input data and the second training input data using the quantized neural network model, and the quantization module is configured to quantize the first training input data and the second training input data using the initial data quantization parameter.

15. The apparatus of claim 14, wherein, parameters of the third operator are obtained by quantizing parameters of the first operator using an operator quantization parameter, and parameters of the fourth operator are obtained by quantizing parameters of the second operator using the operator quantization parameter, the operator quantization parameter being determined according to a range of the parameters of the first operator and a range of the parameters of the second operator.

16. The apparatus of any one of claims 13-15, wherein, The quantized neural network model further includes a compression module configured to compress the output of the third operator and the output of the fourth operator according to a shift parameter, the shift parameter being used to indicate a position of a highest bit in data after the compression in data before the compression, and the second operation module is configured to perform the first operation on the compressed data. The offset parameter is determined according to the number of significant digits of the first training operation data and the number of significant digits of the second training operation data, the first training operation data is obtained by processing the first training input data quantized by using the data quantization parameter by using the third operator, and the second training operation data is obtained by processing the second training input data quantized by using the data quantization parameter by using the fourth operator.

17. A computer-readable storage medium, characterized in that, The computer readable storage medium stores program code for execution by a device, which when executed by the device, causes the device to perform the method of any one of claims 1 to 8.

18. A chip, characterized by The chip comprises a processor and a data interface, and the processor reads instructions stored on a memory through the data interface to execute the method of any one of claims 1 to 8.

Images