A method, device, equipment and storage medium for constructing a neural network model

Abstract

The present invention relates to a method, device and system for constructing a neural network model, comprising: obtaining a preset neural network model; adding a pseudo quantization layer after a hidden layer of the preset neural network model to obtain a new neural network model; the pseudo quantization layer is used to quantize data of a first data type output by the hidden layer into data of a second data type, and perform rounding calculation on the data of the second data type, and dequantize the data of the second data type after the rounding calculation into data of the first data type; based on a target task, the new neural network model is trained to obtain a converged neural network model, thereby reducing the occupied space of the neural network model file, reducing the memory bandwidth, reducing the need for computing resources, obtaining higher performance, and achieving low-power operation without causing much impact on the training results of the neural network model.

Description

Technical Field

[0001] The present disclosure relates to the technical field of machine learning, and in particular to a method, device, equipment and storage medium for constructing a neural network model. Background Art

[0002] When developing neural networks on the computer side, 32-bit floating-point types are generally used to perform numerical calculations of network flows and store network weights, and the computer side can use GPUs with a large number of floating-point computing units to accelerate neural networks. However, in the era of mobile Internet, more and more AI algorithms need to be implemented on mobile terminals, and the efficiency of neural network models that originally ran on PCs has become a major bottleneck on mobile devices with limited resources.

[0003] In order to better implement AI algorithms, it is necessary to propose a solution that can adapt to the running of AI algorithms on terminal devices with limited resources. Summary of the invention

[0004] The present disclosure proposes a method, device, equipment and storage medium for constructing a neural network model, which can compress the size of the neural network model file and reduce the space occupied by the neural network model file. The present disclosure is specifically implemented by the following technical solutions:

[0005] In one aspect, the present disclosure provides a method for constructing a neural network model, comprising:

[0006] Get the preset neural network model;

[0007] A pseudo quantization layer is added after the hidden layer of the preset neural network model to obtain a new neural network model; the pseudo quantization layer is used to quantize the data of the first data type output by the hidden layer into data of the second data type, perform rounding calculation on the data of the second data type, and dequantize the data of the second data type after the rounding calculation into data of the first data type;

[0008] Based on the target task, the new neural network model is trained to obtain a converged neural network model.

[0009] In another aspect, the present disclosure provides a device for constructing a neural network model, comprising:

[0010] A first acquisition module, used to acquire a preset neural network model;

[0011] A pseudo-quantization layer adding module, used for adding a pseudo-quantization layer after the hidden layer of the preset neural network model to obtain a new neural network model; the pseudo-quantization layer is used to quantize the data of the first data type output by the hidden layer into data of the second data type, and to perform rounding calculation on the data of the second data type, and to dequantize the data of the second data type after the rounding calculation into data of the first data type;

[0012] The training module is used to train the new neural network model based on the target task to obtain a converged neural network model.

[0013] In some embodiments, the apparatus further comprises:

[0014] A third acquisition module is used to obtain the maximum value, minimum value of the input data and the maximum value and minimum value of the weight of each neuron in the converged neural network model during the training process;

[0015] The second storage module is used to store the maximum value and minimum value of the input data and the maximum value and minimum value of the weight.

[0016] In some embodiments, the apparatus further comprises:

[0017] A fourth acquisition module is used to acquire test input data, where the data type of the test input data is a first data type;

[0018] A quantization module, used for quantizing the test input data into a second data type;

[0019] An input module, used for inputting the test input data quantized into the second data type and the weight of the quantized second data type into the hidden layer of the converged neural network model;

[0020] A hidden layer training module, used for training the hidden layer based on the test input data quantized into the second data type and the weight of the quantized second data type to obtain output data of the hidden layer, wherein the output data is the second data type;

[0021] The inverse quantization module is used to perform inverse quantization processing on the output data of the second data type to obtain output data of the first output type.

[0022] In some embodiments, the hidden layer training module includes:

[0023] a dequantization unit, configured to dequantize, in the hidden layer, the test input data quantized into the second data type and the weight of the quantized second data type into test data of the first data type and the weight of the first data type;

[0024] a processing unit, configured to process, in the hidden layer, the inversely quantized test data of the first data type and the weight of the first data type to obtain output data of the first data type;

[0025] A quantization unit is used to quantize the output data of the first data type into output data of a second data type in the hidden layer, and the output data of the second data type is the output data of the hidden layer.

[0026] In some embodiments, the dequantization unit comprises:

[0027] A dequantization subunit is used to dequantize the test input data quantized to the second data type and the weight of the quantized second data type into test data of the first data type and the weight of the first data type in the hidden layer based on the saved maximum value, minimum value of the input data and the maximum value, minimum value of the weight.

[0028] On the other hand, the present disclosure provides a device for constructing a neural network model, the device comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the method for constructing a neural network model as described above.

[0029] On the other hand, the present disclosure provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the method for constructing a neural network model as described above.

[0030] The present disclosure provides a method, device, equipment and storage medium for constructing a neural network model, by adding a pseudo quantization layer after the hidden layer of the preset neural network model, quantizing the data of the first data type output by the hidden layer to obtain data of the second data type, then performing rounding calculation on the data of the second data type, and finally dequantizing the data of the second data type after the rounding calculation to obtain data of the first data type, so that a converged neural network model can be obtained by training based on the new neural network model. The technical solution provided by the embodiment of the present application discards the details of the decimal representation of the data after the hidden layer output data is quantized through the processing of the pseudo quantization layer, simulates the expression ability of the second data type data for quantitative training, and achieves the reduction of the space occupied by the neural network model file, the reduction of memory bandwidth, the reduction of the need for computing resources, the acquisition of higher performance, and the realization of low-power operation without causing much impact on the training results of the neural network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 It is a flowchart of a method for constructing a neural network model provided by an embodiment of the present invention;

[0033] Figure 2 It is a flow chart of a method for saving a model after a neural network model is successfully constructed, provided by an embodiment of the present invention;

[0034] Figure 3 It is a flow chart of a method for saving training data after a neural network model is successfully constructed, provided by an embodiment of the present invention;

[0035] Figure 4 is a schematic diagram of a training process of a new neural network model provided by an embodiment of the present invention;

[0036] Figure 5 It is a flowchart of a test process based on a new neural network model provided by an embodiment of the present invention;

[0037] Figure 6 It is a flow chart of a new training processing method for a hidden layer in a neural network model provided by an embodiment of the present invention;

[0038] Figure 7 is a schematic diagram of a new neural network model provided by an embodiment of the present invention;

[0039] Figure 8 is another flow chart of a test process based on a new neural network model provided by an embodiment of the present invention;

[0040] Fig. 9 It is a schematic diagram of the calculation steps of the QuantizedRelu layer in a new neural network model provided by an embodiment of the present invention;

[0041] Fig.10 A device for constructing a neural network model provided by an embodiment of the present invention;

[0042] Fig.11 This is a hardware structure block diagram of a server of a heat prediction method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0044] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0045] The following describes the method for constructing a neural network model of the present invention. This specification provides method operation steps as described in the embodiments or flow charts, but more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, in a parallel processor or multi-threaded processing environment).

[0046] The method for constructing a neural network model provided in the embodiments of this specification includes a training process and a testing process. The training process is a process of obtaining an excellent neural network model through supervised learning of a large number of samples, which can be understood as a process of using data sets and labels to perform numerical analysis to obtain a network (inductive conclusion ideas). The testing process is a process of flowing input data on an existing model and weighting all neuron excitations to obtain the final output, which can be understood as a process of using the network obtained in the training process to perform numerical analysis on the test set to obtain an output (conclusion). They are introduced below.

[0047] Figure 1 is a flow chart of a method for constructing a neural network model provided by an embodiment of the present invention, such as Figure 1 As shown, the method specifically includes:

[0048] S101: Obtain a preset neural network model.

[0049] In this step, the preset neural network model can be a common model such as an ART network, a LVQ network, a Kohonen network, a Hopfield network, etc., which is not limited here. A neural network model generally includes an input layer, an output layer, and one or more hidden layers (also called intermediate layers).

[0050] S103: Add a pseudo quantization layer after the hidden layer of the preset neural network model to obtain a new neural network model; the pseudo quantization layer is used to quantize the data of the first data type output by the hidden layer into data of the second data type, and to perform rounding calculation on the data of the second data type, and to dequantize the data of the second data type after the rounding calculation into data of the first data type.

[0051] In the embodiments of the present specification, the first data type may be high-bit data, such as 32-bit floating-point data, and the second data type may be low-bit data, such as 8-bit integer data. Therefore, step S103 may be to first quantize the high-bit data into low-bit data, then round the result, discard the details of the decimal representation, and finally dequantize the rounded low-bit data into high-bit data, thereby simulating the expression ability of low-bit data for quantization training. This method is closer to the expression ability of the quantization network than pure floating-point number training.

[0052] S105: Based on the target task, the new neural network model is trained to obtain a converged neural network model.

[0053] The target task can be any target task that the training process hopes to achieve, and is not limited here.

[0054] Steps S101-S105 describe the training process.

[0055] The embodiments of the present specification discard the details of the decimal representation of the discrete values ​​after quantization by setting a pseudo quantization layer, simulate the expression ability of low-bit data for quantization training, and train a converged neural network model faster than training with high-bit data.

[0056] S107: Save the data of the new neural network model.

[0057] After the training is completed, the data of the converged neural network model obtained through training is saved so that the new neural network model can be used for testing (forward deduction) later.

[0058] For step S107, in the embodiment of this specification, the data of the new neural network model itself can be saved. The specific steps are as follows: Figure 2 As shown:

[0059] S201: Obtain the weight corresponding to each neuron of the converged neural network model during the training process.

[0060] After the new neural network training is completed, each neuron has a corresponding weight, and the weights of all neurons in each hidden layer are obtained.

[0061] S203: Count the maximum value and the minimum value of the weight respectively.

[0062] After the weights corresponding to all neurons in each hidden layer are obtained in step S201, the maximum value and the minimum value of the weights corresponding to all neurons in the hidden layer are determined.

[0063] S205: quantizing the data type of the weight of each neuron in the converged neural network model from a first data type to a second data type based on the maximum value and the minimum value of the weight.

[0064] Specifically, based on the maximum and minimum values ​​of the weights corresponding to all neurons in each hidden layer determined in step S203, the data type of the weight corresponding to each neuron in the hidden layer is quantized from the first data type to the second data type.

[0065] S207: Save the quantized weight of the second data type.

[0066] The weight of the second data type quantized in step 205 is saved, thereby saving the new neural network model.

[0067] For step S107, in the embodiment of this specification, the data in the training process of the new neural network model can be saved for quantitative use during testing. The specific steps are as follows: Figure 3 As shown:

[0068] S301: Obtain the maximum value, minimum value of input data and the maximum value, minimum value of weight of each neuron in the converged neural network model during the training process.

[0069] S303: Save the maximum value, minimum value of the input data and the maximum value, minimum value of the weight.

[0070] In the embodiments of this specification, Figure 4 A schematic diagram of the training process of a new neural network model is also shown, and the specific process is as follows.

[0071] Figure 4Two hidden layers are specifically shown in the figure, one is the CONV convolution layer and the other is the ReLU6 version layer. The input data of the training process includes the input data set and the output data set. The training adopts the pseudo-quantization training method, using 32-bit floating-point values ​​for training. During the training process, the maximum and minimum values ​​of each blob (data unit in the neural network) data are recorded, and a fake-quant pseudo-quantization layer is added after each layer. The fake-quant layer converts the floating-point value into a discrete value corresponding to the value range and quantizes it into 255 scales, discarding the details of the decimal representation, and simulating the expression ability of 8-bit data for quantitative training. This method is closer to the expression ability of the quantitative network than pure floating-point training.

[0072] After the model parameters are trained, the maximum and minimum values ​​of the weights of each layer are counted, and the 32-bit floating point parameter values ​​(weight values) are converted into 8-bit integers through the quantization formula (the formula is as follows) and saved. The maximum and minimum values ​​of the input data of each blob during the training process, as well as the maximum and minimum values ​​of the network parameters are saved for quantization during forward derivation. The network parameters can be weights or weights and bias items. Among them, the statistics of the maximum / minimum values ​​discard some maximum and minimum values ​​that appear less frequently according to the distribution of weights, and obtain more realistic maximum / minimum values ​​to obtain better quantization effects.

[0073] The quantization formula for quantizing 32-bit floating-point data to 8-bit integer is as follows: the maximum value max and the minimum value min of the data are confirmed according to the sample, and the value range is determined according to min and max, and quantized to 255 values ​​represented by 8-bit.

[0074] u8=(f 32 -min)*255 / (max-min).

[0075] In the embodiments of this specification, Figure 5 A flowchart of a test process based on a new neural network model is also provided, and the specific process includes:

[0076] S501: Acquire test input data, where the data type of the test input data is a first data type.

[0077] The test input data includes an input data set, such as 32-bit floating point data.

[0078] S503: Quantize the test input data into a second data type.

[0079] The second data type is, for example, an 8-bit integer, and the formula used in the quantization process is: u8=(f32 -min)*255 / (max-min).

[0080] S505: Input the test input data quantized into the second data type and the weight of the quantized second data type into the hidden layer of the converged neural network model.

[0081] The quantized weight of the second data type is the weight saved in step S207.

[0082] S507: The hidden layer is trained based on the test input data quantized into the second data type and the weight of the quantized second data type to obtain output data of the hidden layer, where the output data is the second data type.

[0083] The training process includes but is not limited to: convolution, matrix multiplication, activation function, downsampling, concatenation, quantization, dequantization and other computational processes.

[0084] S509: Perform inverse quantization processing on the output data of the second data type to obtain output data of the first output type.

[0085] During the entire testing process, the data transmitted between hidden layers are all of the second data type, thereby reducing the resources required during the transmission process.

[0086] In the embodiment of the present specification, a new training method for the hidden layer in the neural network model is also provided for step S507. The specific flow chart is as follows: Figure 6 shown.

[0087] S601: In the hidden layer, dequantize the test input data quantized into the second data type and the weight of the quantized second data type into test data of the first data type and the weight of the first data type.

[0088] In the hidden layer, based on the saved maximum and minimum values ​​of the input data and the maximum and minimum values ​​of the weights, the test input data quantized to the second data type and the quantized weights of the second data type are dequantized to test data of the first data type and the weights of the first data type.

[0089] Dequantizing the test input data quantized to the second data type and the weight of the quantized second data type is to facilitate calculation using the actual data size, thereby improving the accuracy of the result.

[0090] S603: In the hidden layer, processing is performed based on the inverse-quantized test data of the first data type and the weight of the first data type to obtain output data of the first data type.

[0091] The processing includes but is not limited to: convolution, matrix multiplication, activation function, downsampling, splicing, etc. Here, taking only one hidden layer and the function y=wx as an example, the weight of the first data type after inverse quantization is w, the test data inversely quantized to the first data type is x, and the calculated output data y is also the first data type.

[0092] S605: In the hidden layer, quantize the output data of the first data type into output data of a second data type, and the output data of the second data type is the output data of the hidden layer.

[0093] The quantized output data of the second data type is further transmitted to the next layer.

[0094] If the converged neural network model (new neural network model) includes multiple hidden layers, the same steps as steps S601-605 are performed inside the hidden layer, wherein the input data of the next hidden layer is the output data of the upper hidden layer. If there is only one hidden layer, it is directly output as the output value.

[0095] The present invention proposes a model acceleration method for a quantized model, which aims to use smaller numerical storage to represent network weights, reduce memory bandwidth, reduce the need for computing resources, and obtain a higher performance and low power consumption operation solution. Here, it is proposed to use 8-bit integer values ​​to quantize the model, which has obvious quantization effect, can support a large number of embedded DSP devices, and has a wide range of application scenarios.

[0096] Figure 7 is a schematic diagram of a new neural network model provided in the embodiments of this specification, Figure 8 This is another flow chart of a test process based on a new neural network model provided by the embodiment of this specification. Figure 7 , Figure 8 Provide explanation.

[0097] by Figure 7 The example is used as a convergent neural network model obtained through training in the embodiments of this specification. The convergent neural network model includes an input layer, multiple hidden layers, and an output layer. The parameter d is the number of neurons in the input layer, q and p are the number of neurons in the first hidden layer and the last hidden layer, respectively, l is the number of neurons in the output layer, and m in is the connection weight (i.e., weight) between the i-th neuron in the input layer and the k-th neuron in the first hidden layer, w hj is the connection weight between the hth neuron in the last hidden layer and the jth neuron in the output layer, c k and b h are the outputs of the first hidden layer and the last hidden layer neurons respectively.

[0098] like Figure 8 As shown in the figure, the QuantizedRelu layer in the middle refers to the quantized version of the Relu layer. It is preceded and followed by some quantization and dequantization operations, which together serve as a new quantized Relu layer.

[0099] During the forward derivation of the model, the input data is quantized from 32-bit to 8-bit data for storage at the beginning of the network, and circulates in the network as 8-bit data. Each operation layer needs to run 8-bit quantized versions of operations. These operators include convolution, matrix multiplication, activation function, downsampling, and splicing. They accept 8-bit inputs, calculate the corresponding scale and zero based on the maximum and minimum values, and dequantize the 8-bit data to the actual data size for calculation. Finally, the result is quantized to 8-bit data and transmitted to the next layer. Some layers can be rewritten and fused to offset adjacent quantization / dequantization operations and eliminate redundant steps.

[0100] Fig. 9 This is a method provided by the embodiments of this specification. Figure 8 Schematic diagram of the calculation steps of the QuantizedRelu layer shown.

[0101] Q in the picture in is the input of the layer, Q out is the output of the layer, Q in and Qo ut All are quantized 8-bit data, R in To calculate the input data, R out To calculate the output data, Ri n and R out is the value after inverse quantization. in =Sin(Q in -Z in ), Q out =R out / S out +Z out Quantization and dequantization are achieved by the formulas S=scale=(max-min) / 255 and Z=zero=(0-min) / scale.

[0102] In summary, the input and output of the quantized network are still float, but the intermediate calculations are calculated using 8 bits. It converts the weights and internal node operations into 8-bit values ​​for calculation and storage, thereby achieving the purpose of model compression and model acceleration.

[0103] The embodiment of the present invention also provides a device for constructing a neural network model, such as Fig.10As shown, the device comprises:

[0104] The first acquisition module 1001 is used to acquire a preset neural network model.

[0105] The pseudo-quantization layer adding module 1003 is used to add a pseudo-quantization layer after the hidden layer of the preset neural network model to obtain a new neural network model; the pseudo-quantization layer is used to quantize the data of the first data type output by the hidden layer into data of the second data type, and to perform rounding calculation on the data of the second data type, and to dequantize the data of the second data type after the rounding calculation into data of the first data type.

[0106] The training module 1005 is used to train the new neural network model based on the target task to obtain a converged neural network model.

[0107] In some embodiments, the apparatus further comprises:

[0108] The second acquisition module is used to obtain the weight corresponding to each neuron of the converged neural network model during the training process.

[0109] The statistical module is used to count the maximum value and the minimum value of the weight respectively.

[0110] A quantization module is used to quantize the data type of the weight of each neuron in the converged neural network model from a first data type to a second data type based on the maximum value and the minimum value of the weight.

[0111] The first saving module is used to save the quantized weight of the second data type.

[0112] In some embodiments, the apparatus further comprises:

[0113] A third acquisition module is used to obtain the maximum value, minimum value of the input data and the maximum value and minimum value of the weight of each neuron in the converged neural network model during the training process;

[0114] The second storage module is used to store the maximum value and minimum value of the input data and the maximum value and minimum value of the weight.

[0115] In some embodiments, the apparatus further comprises:

[0116] A fourth acquisition module is used to acquire test input data, where the data type of the test input data is a first data type;

[0117] A quantization module, used for quantizing the test input data into a second data type;

[0118] An input module, used for inputting the test input data quantized into the second data type and the weight of the quantized second data type into the hidden layer of the converged neural network model;

[0119] A hidden layer training module, used for training the hidden layer based on the test input data quantized into the second data type and the weight of the quantized second data type to obtain output data of the hidden layer, wherein the output data is the second data type;

[0120] The inverse quantization module is used to perform inverse quantization processing on the output data of the second data type to obtain output data of the first output type.

[0121] In some embodiments, the hidden layer training module includes:

[0122] a dequantization unit, configured to dequantize, in the hidden layer, the test input data quantized into the second data type and the weight of the quantized second data type into test data of the first data type and the weight of the first data type;

[0123] a processing unit, configured to process, in the hidden layer, the inversely quantized test data of the first data type and the weight of the first data type to obtain output data of the first data type;

[0124] A quantization unit is used to quantize the output data of the first data type into output data of a second data type in the hidden layer, and the output data of the second data type is the output data of the hidden layer.

[0125] In some embodiments, the dequantization unit comprises:

[0126] A dequantization subunit is used to dequantize the test input data quantized to the second data type and the weight of the quantized second data type into test data of the first data type and the weight of the first data type in the hidden layer based on the saved maximum value, minimum value of the input data and the maximum value, minimum value of the weight.

[0127] The device and method embodiments in the device embodiments are based on the same inventive concept.

[0128] The present invention also provides a device for constructing a neural network model, the device comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the method for constructing a neural network model as described above.

[0129] The present invention also provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the method for constructing a neural network model as described above.

[0130] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0131] In the embodiments of this specification, the memory can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, application programs required for functions, etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0132] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking running on a server as an example, Fig.11 is a hardware structure block diagram of a server of a heat prediction method provided in an embodiment of the present application. Fig.11As shown, the server 1100 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1110 (the processor 1110 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1130 for storing data, and one or more storage media 1120 (such as one or more mass storage devices) for storing application programs 1123 or data 1122. Among them, the memory 1130 and the storage medium 1120 can be short-term storage or permanent storage. The program stored in the storage medium 1120 may include one or more modules, each of which may include a series of instruction operations on the server. Furthermore, the central processing unit 1110 can be configured to communicate with the storage medium 1120 and execute a series of instruction operations in the storage medium 1120 on the server 1100. The server 1100 may also include one or more power supplies 1160, one or more wired or wireless network interfaces 1150, one or more input and output interfaces 1140, and / or one or more operating systems 1121, such as Windows Server TM, MacOS XTM, Unix TM, Linux TM, Free BSD TM, etc.

[0133] The input / output interface 1140 may be used to receive or send data via a network. The specific example of the network may include a wireless network provided by a communication provider of the server 1100. In one example, the input / output interface 1140 includes a network adapter (Network Interface Controller, NIC), which may be connected to other network devices via a base station so as to communicate with the Internet. In one example, the input / output interface 1140 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0134] It can be understood by those skilled in the art that Fig.11 The structure shown is only for illustration and does not limit the structure of the above electronic device. Fig.11 More or fewer components as shown, or with Fig.11 Different configurations are shown.

[0135] It can be seen from the embodiments of the construction method, device, equipment and storage medium of the neural network model provided by the present invention that the embodiments of this specification add a pseudo quantization layer after the hidden layer of the preset neural network model, quantize the data of the first data type output by the hidden layer to obtain data of the second data type, then perform rounding calculation on the data of the second data type, and finally perform dequantization on the data of the second data type after the rounding calculation to obtain data of the first data type, so that a converged neural network model can be obtained by training based on the new neural network model. The technical solution provided by the embodiment of the present application discards the details of the decimal representation of the data after the hidden layer output data is quantized through the processing of the pseudo quantization layer, simulates the expression ability of the second data type data for quantitative training, and achieves the reduction of the space occupied by the neural network model file, reduces the memory bandwidth, reduces the need for computing resources, obtains higher performance, and achieves low-power operation without causing much impact on the training results of the neural network model.

[0136] The method for constructing a neural network model provided by the present invention has the following technical effects:

[0137] (1) 8-bit numerical storage compresses the model space to only 25% of the original space.

[0138] (2) 8-bit value storage only occupies 25% of the original memory bandwidth, reducing a large amount of memory access for data reading, and can even better utilize the cache, reducing reading pressure and avoiding memory access bottlenecks.

[0139] (3) In the 8-bit quantized version of the operator, although it is still necessary to dequantize back to the actual data size, the quantization / dequantization operations and the recombination between operators allow more data to be calculated on the 8-bit / 16-bit data size, which improves the computing efficiency compared to 32-bit data. Under the same vectorization and multi-core parallel computing unit, greater computing power can be obtained within one clock cycle, while reducing power consumption.

[0140] (4) Pure fixed-point data processing is the only solution for embedded DSP devices without floating-point computing units. Many processors are very friendly to 8-bit computing.

[0141] After experiments, ResNet was used to verify the ImageNet dataset. The experimental results are as follows: the difference in accuracy between the 8-bit quantized version and the 32-bit floating point version is within 2%, which is acceptable.

[0142] In terms of performance, through the MobileNet network, a lightweight model suitable for mobile terminals, forward deduction experiments were carried out on the single-core and single-threaded case of the Qualcomm 835 platform (large core 2.45GHz), with a performance improvement of nearly 30%.

[0143] It should be noted that the sequence of the embodiments of the present invention described above is for description only and does not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0144] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, system and server embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0145] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0146] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a neural network model, characterized in that: The method comprises: Get the preset neural network model; A pseudo quantization layer is added after the hidden layer of the preset neural network model to obtain a new neural network model; the pseudo quantization layer is used to quantize the data of the first data type output by the hidden layer into data of the second data type, perform rounding calculation on the data of the second data type, and dequantize the data of the second data type after the rounding calculation into data of the first data type; Based on the target task, training the new neural network model to obtain a converged neural network model; Acquire test input data, where the data type of the test input data is a first data type; quantizing the test input data into a second data type; Inputting the test input data quantized into the second data type and the weight of the quantized second data type into the hidden layer of the converged neural network model; In the hidden layer, the test input data quantized into the second data type and the weight of the quantized second data type are dequantized into test data of the first data type and the weight of the first data type; In the hidden layer, processing is performed based on the inverse-quantized test data of the first data type and the weight of the first data type to obtain output data of the first data type; In the hidden layer, the output data of the first data type is quantized into output data of a second data type, and the output data of the second data type is the output data of the hidden layer; Dequantizing the output data of the second data type to obtain output data of the first output type; ResNet is used for validation on the ImageNet dataset.

2. The method according to claim 1, characterized in that: After the new neural network model is trained based on the target task to obtain a converged neural network model, the method further includes: Obtaining the weight corresponding to each neuron of the converged neural network model during the training process; Counting the maximum and minimum values ​​of the weights respectively; quantizing the data type of the weight of each neuron in the converged neural network model from a first data type to a second data type based on the maximum value and the minimum value of the weight; Saves the quantized weights of the second data type.

3. The method according to claim 2, characterized in that After the new neural network model is trained based on the target task to obtain a converged neural network model, the method further includes: Obtaining the maximum value, minimum value of input data and the maximum value, minimum value of weight of each neuron in the converged neural network model during the training process; The maximum and minimum values ​​of the input data and the maximum and minimum values ​​of the weight are saved.

4. The method according to claim 3, characterized in that: In the hidden layer, dequantizing the test input data quantized into the second data type and the quantized weight of the second data type into the test data of the first data type and the weight of the first data type includes: In the hidden layer, based on the saved maximum and minimum values ​​of the input data and the maximum and minimum values ​​of the weights, the test input data quantized to the second data type and the quantized weights of the second data type are dequantized to test data of the first data type and the weights of the first data type.

5. A device for constructing a neural network model, characterized in that: The device comprises: A first acquisition module, used to acquire a preset neural network model; A pseudo-quantization layer adding module, used for adding a pseudo-quantization layer after the hidden layer of the preset neural network model to obtain a new neural network model; the pseudo-quantization layer is used to quantize the data of the first data type output by the hidden layer into data of the second data type, and to perform rounding calculation on the data of the second data type, and to dequantize the data of the second data type after the rounding calculation into data of the first data type; A training module, used to train the new neural network model based on the target task to obtain a converged neural network model; A fourth acquisition module is used to acquire test input data, where the data type of the test input data is a first data type; A quantization module, used for quantizing the test input data into a second data type; An input module, used for inputting the test input data quantized into the second data type and the weight of the quantized second data type into the hidden layer of the converged neural network model; A hidden layer training module, used for training the hidden layer based on the test input data quantized as the second data type and the weight of the second data type after quantization to obtain output data of the hidden layer, wherein the output data is the second data type; the hidden layer training module includes: a dequantization unit, used for dequantizing the test input data quantized as the second data type and the weight of the second data type after quantization into test data of the first data type and the weight of the first data type in the hidden layer; a processing unit, used for processing in the hidden layer based on the dequantized test data of the first data type and the weight of the first data type to obtain output data of the first data type; a quantization unit, used for quantizing the output data of the first data type into output data of the second data type in the hidden layer, wherein the output data of the second data type is the output data of the hidden layer; a dequantization module, configured to perform dequantization processing on the output data of the second data type to obtain output data of the first output type; ResNet is used for validation on the ImageNet dataset.

6. The device according to claim 5, characterized in that The device also includes: A second acquisition module is used to obtain the weight corresponding to each neuron of the converged neural network model during the training process; A statistical module, used to count the maximum and minimum values ​​of the weights respectively; A quantization module, used for quantizing the data type of the weight of each neuron in the converged neural network model from a first data type to a second data type based on the maximum value and the minimum value of the weight; The first saving module is used to save the quantized weight of the second data type.

7. The device according to claim 6, characterized in that The device also includes: A third acquisition module is used to obtain the maximum value, minimum value of the input data and the maximum value and minimum value of the weight of each neuron in the converged neural network model during the training process; The second storage module is used to store the maximum value and minimum value of the input data and the maximum value and minimum value of the weight.

8. The device according to claim 7, characterized in that The dequantization unit comprises: A dequantization subunit is used to dequantize the test input data quantized to the second data type and the weight of the quantized second data type into test data of the first data type and the weight of the first data type in the hidden layer based on the saved maximum value, minimum value of the input data and the maximum value, minimum value of the weight.

9. A device for constructing a neural network model, characterized in that: The device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method for constructing a neural network model as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method for constructing a neural network model as described in any one of claims 1 to 4.

Images