Information processing method and terminal device

Abstract

The disclosed embodiment discloses an information processing method and a terminal device, wherein the method includes: obtaining first information, the first information being information to be processed by the terminal device; invoking a computing instruction in a computing device to calculate the first information to obtain second information; and outputting the second information. By using the disclosed embodiment, the computing device of the terminal device can be used to call a computing instruction to process the first information to output the second information of the target desired by the user, thereby improving the information processing efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of information processing technology, and in particular to information processing methods and terminal devices. Background Art

[0002] With the continuous development of information technology and people's growing needs, people's requirements for the timeliness of information are becoming increasingly higher. Currently, terminal devices obtain information based on general-purpose processors. For example, a set application is run on the general-purpose processor to obtain the current location of an object, or a set application is run on the general-purpose processor to obtain the user's current scene (such as indoors or outdoors). However, this method of obtaining information based on software programs running on a general-purpose processor is limited by the operating speed of the general-purpose processor. Especially when the general-purpose processor is heavily loaded, the information acquisition efficiency is low and the latency is long. Summary of the Invention

[0003] The disclosed embodiments provide an information processing method and a terminal device, which can utilize a computing device in the terminal device to intelligently process first information to output target information desired by the user, thereby improving information processing efficiency.

[0004] The present disclosure provides a computing device for performing machine learning calculations of a machine learning model, the computing device comprising: an operation unit and a controller unit; the operation unit comprising: a master processing circuit and a plurality of slave processing circuits;

[0005] The controller unit is used to obtain input data and calculation instructions;

[0006] The controller unit is further configured to parse the calculation instruction to obtain a plurality of operation instructions, and send the plurality of operation instructions and the input data to the main processing circuit;

[0007] The master processing circuit is configured to perform pre-processing on the input data and transmit data and operation instructions to and from the plurality of slave processing circuits;

[0008] The plurality of slave processing circuits are configured to perform intermediate operations in parallel based on the data and operation instructions transmitted from the master processing circuit to obtain a plurality of intermediate results, and transmit the plurality of intermediate results to the master processing circuit;

[0009] The main processing circuit is used to perform subsequent processing on the multiple intermediate results to obtain the calculation result of the calculation instruction.

[0010] An embodiment of the present application provides a machine learning computing device, comprising one or more computing devices described in the first aspect. The machine learning computing device is configured to obtain data to be computed and control information from other processing devices, perform specified machine learning operations, and transmit the execution results to other processing devices via an I / O interface.

[0011] When the machine learning computing device includes a plurality of computing devices, the plurality of computing devices may be linked and data may be transmitted between the plurality of computing devices via a specific structure;

[0012] Among them, multiple computing devices are interconnected and transmit data through a PCIE bus to support larger-scale machine learning operations; multiple computing devices share the same control system or have their own control systems; multiple computing devices share memory or have their own memory; the interconnection method of multiple computing devices is any interconnection topology.

[0013] An embodiment of the present application provides a combined processing device, which includes a machine learning processing device, a universal interconnection interface, and other processing devices. The machine learning computing device interacts with the other processing devices to jointly complete user-specified operations. The combined processing device may also include a storage device, which is respectively connected to the machine learning computing device and the other processing devices, and is used to store data from the machine learning computing device and the other processing devices.

[0014] An embodiment of the present application provides a neural network chip, which includes the computing device described in the first aspect, the machine learning operation device described in the second aspect, or the combined processing device described above.

[0015] An embodiment of the present application provides a neural network chip packaging structure, which includes the above-mentioned neural network chip;

[0016] An embodiment of the present application provides a board card, which includes the above-mentioned neural network chip packaging structure.

[0017] An embodiment of the present application provides an electronic device, which includes the neural network chip or the above-mentioned board described in the sixth aspect.

[0018] The present application also provides a method for executing a machine learning model. The method is applied to a computing device for executing machine learning calculations. The computing device includes: a computing unit and a controller unit. The computing unit includes: a master processing circuit and multiple slave processing circuits. The method includes:

[0019] The controller unit acquires data, a machine learning model, and a calculation instruction; the controller unit parses the calculation instruction to obtain a plurality of operation instructions, and sends the plurality of operation instructions and the data to the main processing circuit; the main processing circuit performs pre-processing on the data and transmits data and operation instructions between the plurality of slave processing circuits; the plurality of slave processing circuits perform intermediate operations in parallel based on the data and operation instructions transmitted from the main processing circuit to obtain a plurality of intermediate results, and transmit the plurality of intermediate results to the main processing circuit; the main processing circuit performs subsequent processing on the plurality of intermediate results to obtain the calculation results of the calculation instruction.

[0020] In some embodiments, the electronic device includes a data processing device, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a mobile phone, a driving recorder, a navigator, a sensor, a camera, a server, a cloud server, a camera, a camcorder, a projector, a watch, a headset, a mobile storage, a wearable device, a vehicle, a household appliance, and / or a medical device.

[0021] In some embodiments, the means of transportation include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; and the medical equipment include magnetic resonance imaging (MRI), ultrasound machines and / or electrocardiographs.

[0022] The present disclosure provides an information processing method, which is applied to a terminal device provided with a computing device, wherein the computing device stores an instruction set including at least one operation instruction, and the method includes:

[0023] Acquire first information, where the first information is information to be processed by the terminal device;

[0024] Invoking a computing instruction in the computing device to process the first information, thereby obtaining second information;

[0025] The second information is output.

[0026] In some possible embodiments, obtaining the first information includes: preprocessing the obtained original information to obtain the first information, the first information is information with a preset format, and the preprocessing includes at least one of the following: data deduplication processing, data encoding processing, data conversion processing, and normalization processing.

[0027] In some possible embodiments, the operation instruction includes at least one of the following: matrix multiplication vector instruction, vector multiplication matrix instruction, matrix multiplication scalar instruction, tensor operation instruction, matrix addition instruction, matrix subtraction instruction, matrix retrieval instruction, matrix loading instruction, matrix storage instruction and matrix transfer instruction.

[0028] In some possible embodiments, when the first information is voice information, invoking a computing instruction in the computing device to process the first information to obtain the second information includes:

[0029] Invoking a speech recognition algorithm in a computing device to perform speech recognition on the speech information, thereby obtaining second information;

[0030] The second information is text information, and the speech recognition algorithm includes at least one operation instruction for speech recognition.

[0031] In some possible embodiments, when the first information is image information, invoking a computing instruction in the computing device to process the first information to obtain the second information includes:

[0032] Invoking an image style conversion algorithm in a computing device to convert the style of the image information, thereby obtaining second information;

[0033] The second information has a different style from the first information, and the image style conversion algorithm includes at least one operation instruction for image style conversion.

[0034] In some possible embodiments, when the first information is image information and the image information includes at least one object to be identified, invoking a computing instruction in the computing device to process the first information to obtain the second information includes:

[0035] An object detection algorithm is called in a computing device to perform object detection on the image information, thereby obtaining second information; wherein the second information at least includes the object position, and the object detection algorithm includes at least one operation instruction for object detection.

[0036] In some possible embodiments, when the first information is language information to be translated, invoking an operation instruction in the computing device to process the first information to obtain the second information includes:

[0037] Invoking a language translation algorithm in a computing device to translate the language information to obtain second information;

[0038] The first information and the second information are different, and the language translation algorithm includes at least one operation instruction for language translation.

[0039] In some possible embodiments, when the first message is a sentence of conversation, the computing device invokes an operation instruction to process the first message, obtaining the second message including a response to the sentence of conversation. The response is required to be logically related to the content of the first message, and the two messages can form a logically coherent conversation. In this embodiment, the temporal combination of multiple first messages and second messages constitutes a meaningful chat conversation, i.e., a chatbot.

[0040] In some possible embodiments, when the first information is a historical record of a user, including product browsing records and personal basic information (age, gender, etc.), an operation instruction is called in the computing device to process the first information, and the second information obtained includes: information about products / services recommended to the user, such as clothes, movies, services, etc.

[0041] The present disclosure provides a data processing device for an interconnect circuit, which is an interconnect node in the interconnect circuit for connecting one or more transaction data sources to one or more transaction data destinations. The data processing device includes: at least one input terminal and at least one output terminal, each of the input terminals including multiple input ports, output ports, at least two multiplexers, and at least one buffer memory; a buffer memory allocation circuit connected to the multiplexer, configured to control the multiplexer to allocate a temporary storage location for input transaction data based on the current state of the buffer memory; a routing circuit connected to the buffer memory, configured to select an output terminal for transaction data in a buffer queue; an arbitration circuit, configured to determine the buffer queue with priority transmission rights and to ensure that multiple transaction data transmissions competing for the same output terminal sequentially obtain output path occupation rights according to a preset arbitration strategy; and a multiplexer circuit, connected to the output port and the output terminal, configured to relay data transmission in the interconnect circuit.

[0042] The data processing device of the interconnection circuit disclosed herein, wherein,

[0043] The buffer memory includes a plurality of storage locations, each storage location is associated with each input port, so that transaction data is temporarily stored in the corresponding storage location before arriving at the input port and being forwarded to the corresponding output port.

[0044] The data processing device of the interconnection circuit disclosed herein, wherein,

[0045] The routing circuit determines the output terminal associated with the destination source according to the address information of the transaction data to be sent in the storage location.

[0046] The data processing device of the interconnection circuit disclosed herein, wherein,

[0047] The storage location includes at least one storage portion, and the storage location of transaction data is allocated by the buffer storage allocation circuit.

[0048] The data processing device of the interconnection circuit disclosed herein, wherein,

[0049] The multiplexer circuit connects the storage part with the output end to establish a transmission path for the transaction data that obtains the output path occupation right.

[0050] The data processing device of the interconnection circuit disclosed herein, wherein,

[0051] The arbitration circuit further comprises a priority register, wherein the priority register stores the buffer queue number that obtains the priority transmission right;

[0052] The arbitration circuit checks the priority register to determine whether the buffer queue obtains the output path occupation right.

[0053] The data processing device of the interconnection circuit disclosed herein, wherein,

[0054] After the arbitration circuit allows the transaction data to obtain the output path occupation right, it checks whether the output end is occupied, and allows the transaction data that obtains the path occupation right to complete transmission when the output end is idle.

[0055] In addition, the present disclosure also provides a data processing method for an interconnect circuit, which uses the data processing device of the interconnect circuit to perform data processing. The data processing method includes the following steps:

[0056] Step 1: The multiplexer circuit receives transaction data;

[0057] Step 2: the buffer storage allocation circuit allocates a temporary storage location for the transaction data;

[0058] Step 3: The routing circuit selects an output terminal for the transaction data;

[0059] Step 4: The arbitration circuit determines the buffer queue with the priority transmission right according to the transmission request of the transaction data, and enables multiple transaction data transmissions competing for the same output port to obtain the output path occupation right in turn according to the preset arbitration strategy;

[0060] Step 5: The multiplexer circuit allocates a transmission path for the transaction data that obtains the right to occupy the data path, and transmits the transaction data to the downstream node of the interconnection circuit.

[0061] The data processing method of the interconnection circuit disclosed herein, wherein said step 4 further comprises the following steps:

[0062] In step 41 , the arbitration circuit allows different buffer queues to obtain the priority transmission right in each cycle in a round-robin manner, or allows another buffer queue to obtain the priority transmission right after all transmissions in one buffer queue are completed.

[0063] In the data processing method for interconnected circuits disclosed herein, step 4 further comprises:

[0064] In step 42, the arbitration circuit determines whether the output end requested by the transaction data that obtains the priority transmission right is occupied. If so, it waits for the arbitration processing of the next cycle. If not, the arbitration circuit checks whether there are multiple transaction data requests competing for the same output end based on the transmission request of the transaction data. If so, the arbitration circuit allows the multiple transaction data competing for the same transmission path to obtain the output path occupation right in turn. If not, execute the step 5.

[0065] The effects of this disclosure are as follows:

[0066] (1) Multiple buffers are set up at each input port, and storage locations can be flexibly allocated according to different input data. Each buffer can be flexibly configured to be associated with different output ports and controlled by the storage allocation circuit;

[0067] (2) Instead of reserving space for prediction data, buffer storage is dynamically allocated, saving storage and power consumption;

[0068] (3) When used to connect a large number of transaction data sources and destinations, the input buffer does not need to be set up separately for each output port. Only a small number of buffers or even two buffers are needed. This can meet the data transmission requirements while saving storage and power consumption, especially in interconnected circuits with a small amount of data communication.

[0069] (3) There is a unified arbitration for the transaction data to be sent at each input end. The arbitration circuit comprehensively considers the data transmission requests of each input end to improve the utilization rate of the data path.

[0070] Therefore, the present disclosure is a data processing device that can select corresponding transmission paths for multiple transaction data arriving at a convergence node in an interconnected circuit according to their destinations, and simultaneously arbitrate data transmission requests competing for the same transmission path at the same time, thereby improving the processing speed of transaction data in the interconnected circuit, achieving good data flow control, and improving the data throughput rate in the interconnected circuit.

[0071] In a second aspect, an embodiment of the present disclosure provides a terminal device, which includes a functional unit for executing the method of the first aspect.

[0072] In a third aspect, an embodiment of the present disclosure provides another terminal device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the methods of the first aspect above when executing the computer program.

[0073] In a fourth aspect, the present disclosure provides a computer-readable storage medium storing program code executed by a computing device, wherein the program code includes instructions for executing any one of the methods described in the first aspect above.

[0074] In a fifth aspect, the present disclosure provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any one of the methods of the first aspect.

[0075] A matrix operation device, for performing matrix operations according to matrix operation instructions, comprising:

[0076] A storage unit for storing a matrix;

[0077] A register unit, configured to store a matrix address, wherein the matrix address is an address at which the matrix is ​​stored in the storage unit;

[0078] The matrix operation unit is used to obtain a matrix operation instruction, obtain a matrix address in the register unit according to the matrix operation instruction, then obtain a corresponding matrix in the storage unit according to the matrix address, and then perform a matrix operation according to the obtained matrix to obtain a matrix operation result.

[0079] Optionally, also include:

[0080] The instruction cache unit is used to store matrix operation instructions to be executed.

[0081] Optionally, also include:

[0082] The instruction processing unit is used to obtain the matrix operation instruction from the instruction cache unit, process the matrix operation instruction, and provide it to the matrix operation unit.

[0083] Optionally, the instruction processing unit includes:

[0084] An instruction fetch module, configured to fetch matrix operation instructions from the instruction cache unit;

[0085] A decoding module, used for decoding the acquired matrix operation instructions;

[0086] The instruction queue is used to sequentially store the decoded matrix operation instructions.

[0087] Optionally, also include:

[0088] A dependency processing unit is used to determine whether the matrix operation instruction and the previous matrix operation instruction access the same matrix before the matrix operation unit obtains the matrix operation instruction. If so, the matrix operation instruction is provided to the matrix operation unit after the previous matrix operation instruction is executed; otherwise, the matrix operation instruction is directly provided to the matrix operation unit.

[0089] Optionally, when the matrix operation instruction and the previous matrix operation instruction access the same matrix, the dependency processing unit stores the matrix operation instruction in a storage queue, and after the previous matrix operation instruction is executed, provides the matrix operation instruction in the storage queue to the matrix operation unit.

[0090] Optionally, the storage unit is also used to store the matrix operation result.

[0091] Optionally, also include:

[0092] The input and output unit is used to store the matrix in the storage unit, or to obtain the matrix operation result from the storage unit.

[0093] Optionally, the storage unit is a high-speed temporary memory.

[0094] Optionally, the matrix operation instruction includes an operation code and at least one operation field, wherein the operation code is used to indicate the function of the matrix operation instruction, and the operation field is used to indicate data information of the matrix operation instruction.

[0095] Optionally, the matrix operation unit includes a matrix addition component, a matrix multiplication component, a matrix scalar multiplication component and a nonlinear operation component.

[0096] Optionally, the matrix operation unit is a multi-pipeline structure, wherein the matrix multiplication component and the matrix scalar multiplication component are in the first pipeline stage, the matrix addition component is in the second pipeline stage, and the nonlinear operation component is in the third pipeline stage.

[0097] A device for performing forward operations of a convolutional neural network, comprising an instruction storage unit, a controller unit, a data access unit, an interconnection module, a master operation module, and a plurality of slave operation modules, wherein:

[0098] The instruction storage unit reads instructions through the data access unit and stores the read instructions;

[0099] The controller unit reads instructions from the instruction storage unit and translates the instructions into control signals for controlling the behavior of other modules, wherein the other modules include the data access unit, the master operation module and the plurality of slave operation modules;

[0100] The data access unit performs data or instruction read and write operations between the external address space and the device;

[0101] The slave operation module is used to implement the convolution operation of input data and convolution kernel in the convolutional neural network algorithm;

[0102] The interconnection module is used for data transmission between the master operation module and the slave operation module. Before the forward operation of the fully connected layer of the neural network begins, the master operation module transmits the input data to each slave operation module through the interconnection module. After the calculation process of the slave operation module is completed, the interconnection module pieced together the output scalars of each slave operation module into an intermediate vector step by step and transmitted it back to the master operation module.

[0103] The main operation module concatenates the intermediate vectors of all input data into an intermediate result and performs subsequent operations on the intermediate result.

[0104] Optionally, the main operation module adds the intermediate result to the bias data and then performs an activation operation.

[0105] Optionally, multiple slave operation modules use the same input data and respective convolution kernels to calculate respective output scalars in parallel.

[0106] Optionally, the activation function active used by the main operation module is any of the nonlinear functions sigmoid, tanh, relu, softmax, or a linear function.

[0107] Optionally, the interconnection module constitutes a data path for continuous or discrete data between the master operation module and the multiple slave operation modules, and the interconnection module is any of the following structures: tree structure, ring structure, grid structure, hierarchical interconnection, and bus structure.

[0108] Optionally, the main operation module includes a first storage unit, a first operation unit, a first data dependency determination unit, and a first storage unit, wherein:

[0109] The first storage unit is used to cache input data and output data used by the main operation module during the calculation process;

[0110] The first computing unit completes various computing functions of the main computing module;

[0111] The data dependency determination unit is the port for the first operation unit to read and write data from the first storage unit, ensuring that there is no consistency conflict in reading and writing data from the first storage unit, and is responsible for reading the input neuron vector from the first storage unit and sending it to the slave operation module through the interconnection module; and

[0112] The intermediate result vectors from the interconnect modules are sent to the first arithmetic unit.

[0113] Optionally, each slave operation module includes a second operation unit, a second data dependency determination unit, a second storage unit, and a third storage unit, wherein:

[0114] The second operation unit receives the control signal from the controller unit and performs arithmetic logic operations;

[0115] The second data dependency determination unit is responsible for the read and write operations on the second storage unit and the third storage unit during the calculation process, ensuring that there is no consistency conflict in the read and write operations on the second storage unit and the third storage unit;

[0116] The second storage unit caches the input data and the output scalar calculated by the slave operation module; and

[0117] The third storage unit caches the convolution kernels required by the slave operation module during the calculation process.

[0118] Optionally, the first and second data dependency determination units ensure that there is no consistency conflict in reading and writing by determining whether there is a dependency between the data of a control signal that has not yet been executed and the data of a control signal that is being executed. If not, the control signal is allowed to be transmitted immediately; otherwise, the control signal is allowed to be transmitted only after all control signals on which the control signal depends are executed.

[0119] Optionally, the data access unit reads at least one of the following from the external address space: input data, bias data, or convolution kernel.

[0120] A method for performing a forward operation of a single-layer convolutional neural network, comprising:

[0121] In step S1, an IO instruction is pre-stored at the first address of the instruction storage unit;

[0122] In step S2, the operation starts. The controller unit reads the IO instruction from the first address of the instruction storage unit. According to the decoded control signal, the data access unit reads all corresponding convolutional neural network operation instructions from the external address space and caches them in the instruction storage unit.

[0123] In step S3, the controller unit then reads the next IO instruction from the instruction storage unit. Based on the decoded control signal, the data access unit reads all the data required by the main operation module from the external address space to the first storage unit of the main operation module.

[0124] In step S4, the controller unit then reads the next IO instruction from the instruction storage unit. Based on the decoded control signal, the data access unit reads the convolution kernel data required by the operation module from the external address space.

[0125] In step S5, the controller unit then reads the next CONFIG instruction from the instruction storage unit. Based on the decoded control signal, the device configures various constants required for the calculation of the neural network layer.

[0126] In step S6, the controller unit then reads the next COMPUTE instruction from the instruction storage unit. Based on the decoded control signal, the master operation module first sends the input data in the convolution window to each slave operation module through the interconnection module, and stores it in the second storage unit of the slave operation module. Then, the convolution window is moved according to the instruction.

[0127] In step S7, according to the control signal interpreted by the COMPUTE instruction, the operation unit of the operation module reads the convolution kernel from the third storage unit and the input data from the second storage unit, completes the convolution operation of the input data and the convolution kernel, and returns the obtained output scalar through the interconnection module;

[0128] In step S8, in the interconnection module, the output scalars returned from the operation modules are pieced together into complete intermediate vectors step by step;

[0129] In step S9, the main operation module obtains the intermediate vector returned by the interconnection module, and the convolution window traverses all input data. The main operation module splices all returned vectors into an intermediate result. According to the control signal interpreted by the COMPUTE instruction, the main operation module reads the bias data from the first storage unit and adds it to the intermediate result through the vector addition unit to obtain the bias result. The activation unit then activates the bias result and writes the final output data back to the first storage unit.

[0130] In step S10, the controller unit then reads the next IO instruction from the instruction storage unit. According to the decoded control signal, the data access unit stores the output data in the first storage unit to the specified address in the external address space, and the operation ends.

[0131] A method for performing a forward operation of a multi-layer convolutional neural network, comprising:

[0132] The method is executed for each layer. After the execution of the previous layer of convolutional neural network is completed, the operation instruction of this layer uses the output data address of the previous layer stored in the main operation unit as the input data address of this layer, and the convolution kernel and bias data addresses in the instruction are changed to the corresponding addresses of this layer.

[0133] A device for performing forward operations of a fully connected layer of an artificial neural network, comprising an instruction storage unit, a controller unit, a data access unit, an interconnection module, a master operation module, and a plurality of slave operation modules, wherein:

[0134] The instruction storage unit reads instructions through the data access unit and stores the read instructions;

[0135] The controller unit reads instructions from the instruction storage unit and translates the instructions into control signals for controlling the behavior of other modules, wherein the other modules include the data access unit, the master operation module and the plurality of slave operation modules;

[0136] The data access unit performs data or instruction read and write operations between the external address space and the device;

[0137] The interconnection module is used to connect the master operation module and the slave operation module;

[0138] The main operation module is used to implement the function activation operation in the fully connected layer algorithm of the artificial neural network;

[0139] The slave operation module is used to implement the multiplication and addition operations of input neurons and weight parameters in the fully connected layer algorithm of the artificial neural network;

[0140] The interconnection module is used for data transmission between the master operation module and the slave operation module. Before the forward operation of the fully connected layer of the neural network begins, the master operation module transmits the input neuron vector to each slave operation module through the interconnection module. After the calculation process of the slave operation module is completed, the interconnection module piece together the output neuron values ​​of each slave operation module step by step into an intermediate result vector and transmits it back to the master operation module for subsequent calculation.

[0141] Optionally, multiple slave operation modules use the same input neuron vector and respective weight vectors to calculate respective output neuron values ​​in parallel, and the weight vector of each slave operation module is the row vector corresponding to the slave operation module in the weight matrix.

[0142] Optionally, the activation function active used by the main operation module is any of the nonlinear functions sigmoid, tanh, relu, softmax, or a linear function.

[0143] Optionally, the main operation module adds a bias to the intermediate result vector and then performs an activation operation.

[0144] Optionally, the interconnection module constitutes a data path for continuous or discrete data between the master operation module and the multiple slave operation modules, and the interconnection module is any of the following structures: tree structure, ring structure, grid structure, hierarchical interconnection, and bus structure.

[0145] Optionally, the main operation module includes a first storage unit, a first operation unit, a first data dependency determination unit, and a first storage unit, wherein:

[0146] The neuron cache unit is used to cache the input data and output data used by the main operation module during the calculation process;

[0147] The first computing unit completes various computing functions of the main computing module;

[0148] The data dependency determination unit is the port for the first operation unit to read and write data from the first storage unit, ensuring that there is no consistency conflict in reading and writing data from the first storage unit, and is responsible for reading the input neuron vector from the first storage unit and sending it to the slave operation module through the interconnection module; and

[0149] The intermediate result vectors from the interconnect modules are sent to the first arithmetic unit.

[0150] Optionally, each slave operation module includes a second operation unit, a second data dependency determination unit, a second storage unit, and a third storage unit, wherein:

[0151] The second operation unit receives the control signal from the controller unit and performs arithmetic logic operations;

[0152] The second data dependency determination unit is responsible for the read and write operations on the second storage unit and the third storage unit during the calculation process, ensuring that there is no consistency conflict in the read and write operations on the second storage unit and the third storage unit;

[0153] The second storage unit caches the data of the input neuron vector and the output neuron value calculated by the slave operation module; and

[0154] The third storage unit caches the weight vector required by the slave operation module during the calculation process.

[0155] Optionally, the first and second data dependency determination units ensure that there is no consistency conflict in reading and writing by determining whether there is a dependency between the data of a control signal that has not yet been executed and the data of a control signal that is being executed. If not, the control signal is allowed to be transmitted immediately; otherwise, the control signal is allowed to be transmitted only after all control signals on which the control signal depends are executed.

[0156] A method for performing a forward operation of a fully connected layer of a single-layer artificial neural network using a device for performing a forward operation of a fully connected layer of an artificial neural network, comprising:

[0157] Step S1.1, storing the initial instruction in the instruction storage unit;

[0158] Step S1.2, read an instruction from the instruction storage unit;

[0159] Step S1.3, decoding the read instruction;

[0160] Step S1.4, performing corresponding operations according to the control signal obtained by decoding;

[0161] Step S1.5, writing the operation result back to the corresponding storage unit.

[0162] A method for performing a forward operation of a fully connected layer of an artificial neural network by a device for performing a forward operation of a fully connected layer of a single-layer artificial neural network comprises:

[0163] In step S2.1, an IO instruction is pre-stored in the instruction storage unit;

[0164] In step S2.2, the operation starts. The controller unit reads the IO instruction from the instruction storage unit. Based on the decoded control signal, the data access unit reads all the corresponding artificial neural network fully connected layer operation instructions from the external address space and stores them in the instruction storage unit.

[0165] In step S2.3, the controller unit then reads the next IO instruction from the instruction storage unit. Based on the decoded control signal, the data access unit reads all the data required by the main operation module from the external address space into the first storage unit of the main operation module.

[0166] In step S2.4, the controller unit then reads the next IO instruction from the instruction storage unit. Based on the decoded control signal, the data access unit reads the weight matrix data required by the operation module from the external address space.

[0167] In step S2.6, the controller unit then reads the next COMPUTE instruction from the instruction storage unit. Based on the decoded control signal, the master operation module first sends the input neuron vector to each slave operation module through the interconnection module and stores it in the second storage unit of the slave operation module.

[0168] In step S2.7, according to the control signal translated from the COMPUTE instruction, the second operation unit of the operation module reads the weight vector from the third storage unit and the input neuron vector from the second storage unit, performs the dot product operation on the weight vector and the input neuron vector, and returns the intermediate result through the interconnection module;

[0169] In step S2.8, in the interconnection module, the intermediate results returned from the operation modules are pieced together step by step to form a complete intermediate result vector;

[0170] In step S2.9, the main operation module obtains the return value from the interconnection module and, based on the control signal translated from the COMPUTE instruction, reads the bias vector from the first storage unit. This is added to the vector returned by the interconnection module via the vector addition unit. The activation unit then activates the result of the addition and writes the final output neuron vector back to the first storage unit.

[0171] In step S2.10, the controller unit then reads the next IO instruction from the instruction storage unit. According to the decoded control signal, the data access unit stores the output neuron vector in the storage unit to the specified address in the external address space, and the operation ends.

[0172] Between step S2.4 and step S2.6, execute:

[0173] In step S2.5, the controller unit then reads the next CONFIG instruction from the instruction storage unit and configures various constants required for calculation of the neural network layer according to the translated control signal.

[0174] A method for performing a forward operation of a fully connected layer of a multi-layer artificial neural network, comprising:

[0175] For each fully connected layer of the artificial neural network, the method is executed. When the fully connected layer of the previous artificial neural network is executed, the operation instruction of the next layer uses the output neuron address of the previous layer stored in the main operation module as the input neuron address of the current layer, and changes the weight address and / or bias address in the instruction to the corresponding address of the current layer.

[0176] A device for performing pooling operations, comprising an instruction storage unit, a controller unit, a data access unit, and an operation module, wherein:

[0177] The instruction storage unit reads instructions through the data access unit and caches the read instructions;

[0178] The controller unit reads instructions from the instruction storage unit, decodes the instructions into control signals for controlling the behavior of the operation modules, and then distributes the control signals to the operation modules;

[0179] The data access unit is used to access the external address space and complete the loading and storage of data;

[0180] The operation module is used to complete the maximum value operation of the maxpooling operation, or to complete the accumulation and multiplication operations of the avgpooling operation.

[0181] The operation module includes an operation unit, a data dependency judgment unit and a neuron storage unit.

[0182] The neuron storage unit is used to cache input data and output data used by the operation module during the calculation process;

[0183] The computing unit is used to complete various computing functions of the computing module;

[0184] The data dependency judgment unit is a port for the operation unit to read and write the neuron storage unit, and is also used to ensure that there is no consistency conflict in reading and writing data in the neuron storage unit.

[0185] For maxpooling, during the forward operation, the operation module compares the size of each input vector in turn, takes the maximum value, and obtains the output vector.

[0186] For maxpooling, during the forward operation, the operation module cyclically reads the input vector of the pooling kernel, performs the above-mentioned size comparison operation, obtains the new kernel output vector, and saves the index vector corresponding to each output vector until the pooling operation of this layer is completed.

[0187] For maxpooling, during reverse training, the operation module outputs the input gradient vector to the corresponding storage location through the data access unit according to the index vector saved during the forward operation to obtain the output gradient vector.

[0188] For avgpooling, during the forward operation, operation module 4 accumulates each input vector in turn; then, operation module 4 performs a multiplication by 1 / kernel_size to obtain an output vector, where kernel_size represents the size of the pooling kernel; the new kernel input vector is read cyclically, and the above accumulation and multiplication operations are performed to obtain a new kernel output vector, until the pooling operation of this layer is completed.

[0189] For avgpooling, during reverse training, the operation module multiplies the input gradient vector by 1 / kernel_size, and outputs the input gradient vector to the corresponding storage location through the data access unit to obtain the output gradient vector.

[0190] The device for performing pooling operations, wherein the data dependency judgment unit is used to judge whether there is a dependency relationship between the data of a control signal that has not yet been executed and the data of a control signal that is being executed. If not, the group of control signals is allowed to be transmitted immediately; otherwise, the group of control signals is allowed to be transmitted only after all control signals on which the control signal depends are fully executed.

[0191] A method for performing a pooling operation on a single-layer artificial neural network comprises the following steps:

[0192] Read in instructions and cache the read instructions;

[0193] Decode the instructions into control signals, and then distribute the control signals to the operation modules that perform pooling operations;

[0194] The operation module performs the maximum value operation of the maxpooling operation, or performs the accumulation and multiplication operations of the avgpooling operation.

[0195] For maxpooling, during the forward operation, the operation module compares the size of each input vector in turn, takes the maximum value, and obtains the output vector.

[0196] For maxpooling, during the forward operation, the operation module cyclically reads the input vector of the pooling kernel, performs the above-mentioned size comparison operation, obtains the new kernel output vector, and saves the index vector corresponding to each output vector until the pooling operation of this layer is completed.

[0197] For maxpooling, during reverse training, the operation module outputs the input gradient vector to the corresponding storage location through the data access unit according to the index vector saved during the forward operation to obtain the output gradient vector.

[0198] For avgpooling, during the forward operation, the operation module accumulates each input vector in turn; then the operation module multiplies by 1 / kernel_size to obtain the output vector, where kernel_size represents the size of the pooling kernel; the new kernel input vector is read in a loop, and the above accumulation and multiplication operations are performed to obtain the new kernel output vector until the pooling operation of this layer is completed.

[0199] For avgpooling, during reverse training, the operation module multiplies the input gradient vector by 1 / kernel_size, and outputs the input gradient vector to the corresponding storage location through the data access unit to obtain the output gradient vector.

[0200] A method for performing pooling operations on a multi-layer artificial neural network,

[0201] When the previous layer in the multi-layer artificial neural network is executed, the operation instructions of the next layer use the output neuron vector or output gradient vector calculated in the operation module as the input neuron vector or input gradient vector for the next layer training to perform the calculation of the above method.

[0202] A device for performing batch normalization operation includes an instruction storage unit, a controller unit, a data access unit and an operation module, wherein:

[0203] The instruction storage unit reads instructions through the data access unit and caches the read instructions;

[0204] The controller unit reads instructions from the instruction storage unit, decodes the instructions into microinstructions that control the behavior of other units or modules, and then distributes the respective microinstructions to each unit or module;

[0205] The data access unit is used to access the external address space to complete the loading and storage of data;

[0206] The operation module is used for the forward process or reverse process of the batch normalization operation.

[0207] The operation module includes an operation unit, a data dependency determination unit, a neuron cache unit and an intermediate value cache unit, wherein:

[0208] The arithmetic unit is used to receive microinstructions issued by the controller unit and perform arithmetic and logical operations;

[0209] The data dependency judgment unit is used to perform read and write operations on the neuron cache unit to ensure that there is no read and write consistency conflict in the data used between instructions;

[0210] The neuron buffer unit is used to buffer input neuron data and output neuron data;

[0211] The intermediate value cache unit is used to cache the intermediate value data required during the calculation process of the operation module.

[0212] The operation unit performs the following calculation process in the forward process of batch normativization operation:

[0213] y=f(x)=alpha*(xE[x]) / sqrt(var(x)+eps)+beta, where x is the input neuron data, y is the output neuron data, alpha and beta are learning parameters, which are continuously updated during the reverse training process and used in the formula for calculating the output neuron data y; the minimum constant eps, the mean E[x] represents the mean of the neuron data x of the input data with the batch size as a total amount, and var[x] represents the variance of the corresponding input neuron data x with the batch size as a total amount.

[0214] The operation unit performs the following calculation process in the reverse process of the batch normativization operation:

[0215] Assume that the gradient input to a pixel point is dl / dY, the gradient output in the reverse direction is dl / dx, the output of the forward process is Y, and the meaning of the remaining parameters is the same as the forward process. Then the gradient back-propagated through batch normalization is dl / dx = (alpha / sqrt(var(x)+eps))*(dl / dY-mean(dl / dY)-mean(dl / dY*Y)*Y), where mean is the mean operation. The gradient of the learning parameter alpha is: dl / dalpha = (Σdl / dY)*Y, and the gradient of the learning parameter beta is: dl / dbeta = Σdl / dY. The values ​​of the learning parameters are updated by these two gradients.

[0216] A method for performing a batch normalization operation comprises the following steps:

[0217] Using an instruction storage unit to read instructions and cache the read instructions;

[0218] Decoding the instruction into a microinstruction for controlling a computing module;

[0219] The operation module is used to perform a forward process or a reverse process of a batch normalization operation.

[0220] The operation module utilizes a neuron cache unit to cache input neuron data and output neuron data, and utilizes an intermediate value cache unit to cache intermediate value data required in the calculation process.

[0221] The operation unit performs the following calculation process in the forward process of batch normativization operation:

[0222] The operation unit performs the following calculation process in the forward process of batch normativization operation:

[0223] y=f(x)=alpha*(xE[x]) / sqrt(var(x)+eps)+beta, where x is the input neuron data, y is the output neuron data, alpha and beta are learning parameters, which are continuously updated during the reverse training process and used in the formula for calculating the output neuron data y; the minimum constant eps, the mean E[x] represents the mean of the neuron data x of the input data with the batch size as a total amount, and var[x] represents the variance of the corresponding input neuron data x with the batch size as a total amount.

[0224] Assume that the gradient input to a pixel point is dl / dY, the gradient output in the reverse direction is dl / dx, the output of the forward process is Y, and the meaning of the remaining parameters is the same as the forward process. Then the gradient back-propagated through batch normalization is dl / dx = (alpha / sqrt(var(x)+eps))*(dl / dY-mean(dl / dY)-mean(dl / dY*Y)*Y), where mean is the mean operation. The gradient of the learning parameter alpha is: dl / dalpha = (Σdl / dY)*Y, and the gradient of the learning parameter beta is: dl / dbeta = Σdl / dY. The values ​​of the learning parameters are updated by these two gradients.

[0225] A device for performing neural network operations and matrix / vector operations, comprising a storage unit, a register unit, a control unit, an operation unit, and a high-speed temporary storage memory, wherein:

[0226] Storage unit, used to store neurons / matrices / vectors;

[0227] A register unit, configured to store a neuron address / matrix address / vector address, wherein the neuron address is the address of a neuron stored in the storage unit, the matrix address is the address of a matrix stored in the storage unit, and the vector address is the address of a vector stored in the storage unit;

[0228] A control unit, used to perform decoding operations and control each unit module according to the read instructions;

[0229] an operation unit, configured to obtain a neuron address / matrix address / vector address from the register unit according to an instruction, obtain a corresponding neuron / matrix / vector from the storage unit according to the neuron address / matrix address / vector address, and perform an operation based on the obtained neuron / matrix / vector and / or data carried in the instruction to obtain an operation result;

[0230] The method is characterized in that the neuron / matrix / vector data involved in the calculation of the operation unit is temporarily stored in a high-speed temporary memory, and the operation unit reads the data from the high-speed temporary memory when needed.

[0231] The scratchpad memory can support neuron / matrix / vector data of different sizes.

[0232] The register unit is a scalar register, which provides the scalar register required in the operation process.

[0233] The arithmetic unit includes a vector multiplication component, an accumulation component and a scalar multiplication component; and

[0234] The operation unit is responsible for the neural network / matrix / vector operations of the device, including convolutional neural network forward operation, convolutional neural network training operation, neural network Pooling operation, full connection neural network forward operation, full connection neural network training operation, batch normalization operation, RBM neural network operation, matrix-vector multiplication operation, matrix-matrix addition / subtraction operation, vector outer product operation, vector inner product operation, vector arithmetic operation, vector logic operation, vector transcendental function operation, vector comparison operation, vector maximum / minimum value operation, vector circular shift operation, and random vector operation that generates a certain distribution.

[0235] The apparatus further comprises an instruction cache unit for storing operation instructions to be executed; the instruction cache unit is preferably a reorder cache; and

[0236] The device also includes an instruction queue for sequentially caching decoded instructions and sending them to the dependency processing unit.

[0237] The device also includes a dependency processing unit and a storage queue. The dependency processing unit is used to determine whether the operation instruction and the previous operation instruction access the same neuron / matrix / vector storage address before the operation unit obtains the instruction. If so, the operation instruction is stored in the storage queue; otherwise, the operation instruction is directly provided to the operation unit, and after the previous operation instruction is executed, the operation instruction in the storage queue is provided to the operation unit; the storage queue is used to store instructions that have data dependencies with previous instructions, and submit the instructions after the dependencies are eliminated.

[0238] The instruction set of the device adopts a Load / Store structure, and the arithmetic unit does not operate on data in the memory; and

[0239] The instruction set of the device preferably adopts a very long instruction word architecture and preferably adopts fixed-length instructions.

[0240] The operation instruction executed by the operation unit includes at least one operation code and at least three operands; wherein, the operation code is used to indicate the function of the operation instruction, and the operation unit performs different operations by identifying one or more operation codes; the operand is used to indicate the data information of the operation instruction, wherein the data information is an immediate number or a register number.

[0241] Preferably, when the operation instruction is a neural network operation instruction, the neural network operation instruction includes at least one operation code and 16 operands;

[0242] Preferably, when the operation instruction is a matrix-matrix operation instruction, the matrix-matrix operation instruction includes at least one operation code and at least four operands;

[0243] Preferably, when the operation instruction is a vector operation instruction, the vector operation instruction includes at least one operation code and at least three operands;

[0244] Preferably, when the operation instruction is a matrix-vector operation instruction, the matrix-vector operation instruction includes at least one operation code and at least 6 operands.

[0245] A device for performing neural network operations and matrix / vector operations, comprising:

[0246] The instruction fetch module is used to fetch the next instruction to be executed from the instruction sequence and pass it to the decoding module;

[0247] A decoding module, configured to decode the instruction and transmit the decoded instruction to an instruction queue;

[0248] An instruction queue, used to sequentially cache instructions decoded by the decoding module and send them to the dependency processing unit;

[0249] Scalar register, used to provide scalar register for operation;

[0250] A dependency processing unit, configured to determine whether a current instruction has a data dependency relationship with a previous instruction, and if so, store the current instruction in a storage queue;

[0251] A storage queue, configured to cache a current instruction that has a data dependency relationship with a previous instruction, and to transmit the current instruction after the dependency relationship between the current instruction and the previous instruction is resolved;

[0252] a reorder cache for caching instructions as they are executed, and after execution, determining whether the instruction is the earliest instruction among the uncommitted instructions in the reorder cache, and if so, committing the instruction;

[0253] The arithmetic unit is used to perform all neural network operations and matrix / vector operations;

[0254] A high-speed temporary memory for temporarily storing neuron / matrix / vector data involved in the calculation of the arithmetic unit, and the arithmetic unit reading from the high-speed temporary memory when needed; the high-speed temporary memory is preferably capable of supporting data of different sizes;

[0255] The IO memory access module is used to directly access the high-speed temporary memory and is responsible for reading or writing data from the high-speed temporary memory.

[0256] A method for performing neural network operations and matrix / vector instructions, comprising the following steps:

[0257] Step S1: The instruction fetch module fetches a neural network operation and matrix / vector instruction and sends the instruction to the decoding module;

[0258] Step S2: the decoding module decodes the instruction and sends the instruction to the instruction queue;

[0259] Step S3, in the decoding module, the instruction is sent to the instruction receiving module;

[0260] Step S4: the instruction receiving module sends the instruction to the microinstruction decoding module for microinstruction decoding;

[0261] Step S5: The microinstruction decoding module obtains the neural network operation operation code and neural network operation operand of the instruction from the scalar register, and decodes the instruction into microinstructions for controlling various functional components, and sends them to the microinstruction issue queue;

[0262] Step S6: After obtaining the required data, the instruction is sent to the dependency processing unit; the dependency processing unit analyzes whether the instruction has a data dependency with the previously executed instruction. If so, the instruction needs to wait in the storage queue until it no longer has a data dependency with the previously executed instruction.

[0263] Step S7, sending the microinstruction corresponding to the instruction to the computing unit;

[0264] In step S8, the operation unit retrieves the required data from the high-speed temporary memory according to the address and size of the required data, and then completes the neural network operation and / or matrix / vector operation corresponding to the instruction in the operation unit.

[0265] A data publishing device based on a fractal tree network structure, comprising:

[0266] a central node, which is a communication data center of the on-chip network and is used to broadcast or multicast communication data to multiple leaf nodes;

[0267] A plurality of leaf nodes, which are communication data nodes of the on-chip network and are used to transmit communication data to the central leaf node;

[0268] A forwarder module, configured to connect the central node and the plurality of leaf nodes, and forward communication data through the forwarding module;

[0269] The multiple leaf nodes are divided into N groups, and the number of leaf nodes in each group is the same. The central node communicates with each group of leaf nodes separately through the repeater module. The communication structure formed by each group of leaf nodes has self-similarity. The multiple leaf nodes and the central node are communicated with each other in a complete multi-branch tree manner through multiple layers of the repeater module.

[0270] Each node includes a local cache structure for storing a subset of the data published by the central node;

[0271] Each leaf node has an ID, and the IDs are numbered in order from the topology side of the complete multi-branch tree.

[0272] The data distribution devices share a clock signal.

[0273] The forwarder module includes a local cache structure for storing data.

[0274] A data publishing method using the data publishing device publishes communication data to the multiple leaf nodes through the central node, wherein, after the data sender is ready to send data, it sends a data valid signal and places the data on the bus; after the data receiver is ready to receive data, it sends a data ready to receive signal; when both the data valid signal and the data ready to receive signal are detected, the data sender believes that the data has been sent and received by the data receiver.

[0275] When the communication data is broadcast from the central node to the multiple leaf nodes, the data first enters the local cache of the forwarder module directly connected to the central node from the central node through the handshake protocol for temporary storage. After each handshake protocol succeeds, the data enters the local cache of the next-level intermediate forwarding module for temporary storage, and finally enters the forwarder module directly connected to the leaf node, and is published by the forwarder module to a group of leaf nodes connected to it.

[0276] If the handshake protocol between the data sender and the data receiver is successful in the next clock beat, the data is stored in the local cache of the data receiver in a pipeline manner; if the handshake protocol is unsuccessful, the data is saved in the local cache of the current layer, and the current layer is made the data receiver of the previous layer, and stops sending data to prepare for receiving signals, so that the data in the local cache of the current layer stops updating, and the data is kept in the current layer until the handshake protocol is successful.

[0277] When the central node multicasts communication data to the multiple leaf nodes, the data first enters the local cache of the repeater module directly connected to the central node from the central node through the handshake protocol for temporary storage. After each handshake protocol succeeds, the data enters the local cache of the next layer of intermediate repeater module for temporary storage, and finally enters the repeater module directly connected to the leaf node, and is published by the repeater module to a group of leaf nodes connected to it.

[0278] When receiving data, the leaf node selects data of a preset bandwidth according to the ID corresponding thereto.

[0279] A sparsely connected artificial neural network computing device, comprising:

[0280] A mapping unit, configured to convert input data into a storage format in which input neurons and weights correspond one to one, and store the data in a storage device and / or a cache;

[0281] a storage device for storing data and instructions;

[0282] An operation unit is used to perform corresponding operations on the data according to the instructions stored in the storage device; the operation unit mainly performs three operations, the first step is to multiply the input neurons and weight data; the second step is to perform an addition tree operation, which is used to add the weighted output neurons processed in the first step through the addition tree step by step, or to add the output neurons and the bias to obtain the biased output neurons; the third step is to perform an activation function operation to obtain the final output neurons.

[0283] The one-to-one correspondence in the mapping unit is expressed as follows:

[0284] First scenario:

[0285] 1 indicates a connection, 0 indicates no connection, and the connection status of each output and all inputs is composed of a string of 0s and 1s to represent the connection relationship of the output; or

[0286] 1 indicates a connection, 0 indicates no connection, and the connection status of each input and all outputs is composed of a string of 0s and 1s to represent the connection relationship of the input;

[0287] Second scenario:

[0288] The connection relationship of an output is represented by the distance from the location of the first connection of the output to the first input neuron, the distance from the second input neuron of the output to the previous input neuron, the distance from the third input neuron of the output to the previous input neuron, and so on, until all the inputs of the output are exhausted.

[0289] The artificial neural network computing device further includes a DMA for reading and writing data or instructions in the storage device and the cache.

[0290] The artificial neural network computing device further comprises:

[0291] an instruction cache for storing specialized instructions; and

[0292] The control unit is used to read the dedicated instructions from the instruction cache and decode them into instructions for each operation unit.

[0293] The artificial neural network computing device further comprises:

[0294] an input neuron buffer, for buffering input neuron data input to the arithmetic unit; and

[0295] Weight cache, used to cache weight data.

[0296] The artificial neural network computing device further comprises:

[0297] The output neuron buffer is used to buffer the output neurons output by the computing unit.

[0298] The mapping unit is used to convert the input data into a storage mode in which the input neurons and weights correspond one to one, and output the data to the operation unit instead of storing the data in the storage device.

[0299] The artificial neural network computing device also includes an input neuron cache and / or a weight cache, wherein the input neuron cache is used to cache input neuron data input to the operation unit, and the weight cache is used to cache weight data. The mapping unit is used to convert the input data into a storage method in which input neurons and weights correspond one to one, and output it to the input neuron cache and / or weight cache.

[0300] The activation function executed by the operation unit in the third step is a sigmoid function, a tanh function or a ReLU function.

[0301] A computational method for a sparsely connected artificial neural network comprises the following steps:

[0302] Step 1: Convert the input data into a storage method in which the input neurons and weights correspond one to one; wherein the correspondence is expressed as:

[0303] First scenario:

[0304] 1 indicates a connection, 0 indicates no connection, and the connection status of each output and all inputs is composed of a string of 0s and 1s to represent the connection relationship of the output; or

[0305] 1 indicates a connection, 0 indicates no connection, and the connection status of each input and all outputs is composed of a string of 0s and 1s to represent the connection relationship of the input;

[0306] Second scenario:

[0307] The distance between the first connection location of an output and the first input neuron, the distance between the second input neuron of the output and the previous input neuron, the distance between the third input neuron of the output and the previous input neuron, and so on, until all the inputs of the output are exhausted, to represent the connection relationship of the output

[0308] Step 2, multiply the input neuron and weight data;

[0309] Step 3: perform an addition tree operation to add the weighted output neurons processed in the first step step by step through the addition tree, or add the output neurons and the bias to obtain the biased output neurons;

[0310] Step 4: Execute activation function operation to obtain the final output neuron; wherein the activation function is a sigmoid function, a tanh function, or a ReLU function.

[0311] A neural network processing system, comprising:

[0312] At least one on-chip storage medium for storing data transmitted from outside the neural network processing system or for storing data generated during the processing;

[0313] At least one on-chip address index module, used to map the input index to the correct storage address when performing an operation;

[0314] A multi-core processing module composed of multiple core processing modules is used to perform vector multiplication and addition operations in neural network operations;

[0315] At least one ALU module, configured to obtain input data from the multi-core processing module or the on-chip storage medium to perform nonlinear operations that the multi-core processing module cannot complete,

[0316] The multiple core processing modules share the on-chip storage medium and the ALU module, or the multiple core processing modules have independent on-chip storage media and ALU modules.

[0317] The data generated during the processing includes processing results or intermediate settlement results.

[0318] When the processing system of the neural network performs processing, the same input neuron is sent to the multiple core processing modules respectively, and different input weights are assigned to different core processing modules. The multiple core processing modules respectively perform vector inner product operations on the input neurons and input weights to obtain different output neurons.

[0319] When the neural network processing system performs two-dimensional or multi-dimensional operations, the input feature map is sent to the multiple core processing modules respectively, and the multiple core processing modules respectively process a layer of output feature map.

[0320] When the neural network processing system performs two-dimensional or multi-dimensional operations, the input feature map is sent to the multiple core processing modules respectively, and the multiple core processing modules respectively process different areas of the same output feature map.

[0321] After the multiple core processing modules have completed the processing of the current output feature map respectively, the multi-core processing module then performs new output feature map processing.

[0322] When the neural network processing system performs one-dimensional operations, the same input is sent to the multiple core processing modules respectively. The multiple core processing modules process different output neurons respectively. After the multiple core processing modules complete the processing of the current output neurons respectively, the multi-core processing module then performs the processing of the new input.

[0323] The multiple core processing modules of the multi-core processing module are of homogeneous design or heterogeneous design.

[0324] A method for processing a neural network, the method comprising:

[0325] The on-chip address index module maps the input index to the correct storage address;

[0326] Retrieving input data from an on-chip storage medium according to the storage address;

[0327] Sending the input data to the multi-core processing module or the ALU module;

[0328] The multi-core processing module performs vector multiplication and addition operations in neural network operations, and the ALU module performs nonlinear operations that cannot be completed by the multi-core processing module based on the processing results of the multi-core processing module or the input data obtained from the on-chip storage medium;

[0329] The data generated during the processing is cached in the on-chip storage medium.

[0330] The method further comprises:

[0331] The same input neuron is sent to the multiple core processing modules respectively, and different input weights are allocated to different core processing modules. The multiple core processing modules respectively perform vector inner product operations on the input neurons and input weights to obtain different output neurons.

[0332] A device supporting discrete data representation for executing forward operations of an artificial neural network comprises an instruction cache unit, a controller unit, a data access unit, an interconnection module, a master operation module, and a plurality of slave operation modules, wherein:

[0333] The instruction cache unit is used to read instructions through the data access unit and cache the read instructions;

[0334] The controller unit is used to read instructions from the instruction cache unit and decode the instructions into microinstructions that control the behavior of the interconnection module, the main operation module, and the slave operation module;

[0335] The data access unit is used to write discrete data or continuous data from the external address space to the corresponding data cache units of the master operation module and each slave operation module, or read discrete data or continuous data from the data cache units to the external address space;

[0336] At the beginning of the forward calculation phase of each neural network layer, the master operation module transmits the discrete or continuous input neuron vector of this layer to all the slave operation modules through the interconnection module. After the calculation process of the slave operation module is completed, the interconnection module piece together the discrete or continuous output neuron values ​​of each slave operation module into an intermediate result vector. When the input data is a mixture of discrete data and continuous data, the slave operation module adopts the pre-set corresponding calculation method for different discrete data.

[0337] The main operation module is used to complete subsequent calculations using the intermediate result vector. When the input data is a mixture of discrete data and continuous data, the main operation module adopts a pre-set corresponding calculation method for different discrete data.

[0338] Discrete data representation refers to the representation method of using specific discrete numbers instead of real continuous data.

[0339] The plurality of slave operation modules utilize the same discrete or continuous input neuron vector and respective different discrete or continuous weight vectors to calculate respective discrete or continuous output neuron values ​​in parallel.

[0340] The main operation module performs any of the following operations on the intermediate result vector:

[0341] Add bias operation, add bias to the intermediate result vector;

[0342] Activate the intermediate result vector. The activation function active is any of the nonlinear functions sigmoid, tanh, relu, softmax, or a linear function.

[0343] Sampling operation, compare the intermediate result vector with the random number, output 1 if it is greater than the random number, and output 0 if it is less than the random number; or

[0344] Pooling operations, including maximum pooling or average pooling.

[0345] The slave operation module includes an input neuron cache unit for caching discrete or continuous input neuron vectors.

[0346] The interconnection module constitutes a data path for continuous or discrete data between the master operation module and the plurality of slave operation modules.

[0347] The main operation module includes an operation unit, a data dependency judgment unit, and a neuron cache unit, wherein:

[0348] The neuron cache unit is used to cache the discrete or continuous input and output data used by the main operation module during the calculation process;

[0349] The calculation unit completes various calculation functions of the main calculation module. When the input data is a mixture of discrete data and continuous data, it adopts the corresponding pre-set calculation method for different discrete data.

[0350] The data dependency judgment unit is the port for the operation unit to read and write the neuron cache unit, ensuring that there is no consistency conflict when reading and writing continuous or discrete data in the neuron cache unit. It is also responsible for reading the input discrete or continuous neuron vector from the neuron cache unit and sending it to the slave operation module through the interconnection module; and

[0351] Intermediate result vectors from the interconnect modules are sent to the arithmetic units.

[0352] Each slave operation module includes an operation unit, a data dependency determination unit, a neuron cache unit, and a weight cache unit, wherein:

[0353] The arithmetic unit receives the microinstructions issued by the controller unit and performs arithmetic and logical operations. When the input data is a mixture of discrete data and continuous data, it adopts the corresponding pre-set calculation method for different discrete data;

[0354] The data dependency judgment unit is responsible for the read and write operations of the neuron cache unit supporting discrete data representation and the weight cache unit supporting discrete data representation during the calculation process, ensuring that there is no consistency conflict in the read and write operations of the neuron cache unit supporting discrete data representation and the weight cache unit supporting discrete data representation;

[0355] The neuron cache unit caches the data of the input neuron vector and the output neuron value calculated by the slave operation module; and

[0356] The weight cache unit caches the weight vectors in discrete or continuous representation required by the slave operation module during the calculation process.

[0357] The data dependency judgment unit ensures that there are no consistency conflicts in reading and writing by judging whether there is a dependency between the data of the microinstructions that have not yet been executed and the microinstructions that are being executed. If not, the microinstruction is allowed to be issued immediately. Otherwise, the microinstruction is not allowed to be issued until all the microinstructions that the microinstruction depends on are executed.

[0358] The operation unit in the main operation module or the slave operation module includes an operation decision unit and a mixed data operation unit. When the input data is mixed data, the operation decision unit decides what operation should be performed on the mixed data based on the discrete data therein, and then the mixed data operation unit performs the corresponding operation based on the decision result of the operation decision unit.

[0359] The operation unit in the main operation module or the slave operation module also includes at least one of a discrete data operation unit and a continuous data operation unit, and a data type judgment unit. When the input data is all discrete data, the discrete data operation unit performs corresponding operations based on the input discrete data by looking up the table. When the input data is all continuous data, the continuous data operation unit performs corresponding operations.

[0360] The system further includes a continuous discrete conversion unit, which includes a preprocessing module, a distance calculation module, and a judgment module. Assuming that M discrete data are used, M=2m, m≥1, these discrete data correspond to M values ​​in a predetermined interval [-zone, zone], where:

[0361] The preprocessing module preprocesses the input continuous data x using the clip(-zone, zone) operation to obtain the preprocessed data y in the interval [-zone, zone], where if x ≤ -zone, then y = -zone; if x ≥ zone, then y = zone; if -zone < x < zone, then the preprocessed data y = x;

[0362] The distance calculation module calculates the distance between the pre-processed data y and the above values; and

[0363] The judgment module calculates and outputs discrete data based on the distance.

[0364] Also includes any one or more of the following:

[0365] The predetermined interval [-zone,zone] is [-1,1] or [-2,2];

[0366] The absolute values ​​of the M numbers are reciprocals of powers of 2; or

[0367] Determine module execution:

[0368] Output the discrete data corresponding to the value closest to the pre-processed data y. If there are two values ​​​​with the same distance from the pre-processed data, output the discrete data corresponding to either of them; or

[0369] Calculate the normalized probability of the preprocessed data y to any of the two nearest values, and compare the normalized probability corresponding to any of the two values ​​with the random number z between (0,1) generated by the random number generation module. If z is less than the probability, output the discrete data, otherwise output another discrete data.

[0370] A method for performing a forward operation of a single-layer artificial neural network using a forward operation device, comprising:

[0371] The data access unit reads all artificial neural network operation instructions related to the forward operation of the artificial neural network at this layer from the external address space and caches them in the instruction cache unit;

[0372] The continuous-discrete conversion module reads the continuous data that needs to be converted by the neural network layer from the external address space, converts it into discrete data, and then stores it back into the external address space;

[0373] The data access unit reads all discrete or continuous data related to the forward operation of the artificial neural network layer required by the main operation module from the external address space to the neuron cache unit of the main operation module;

[0374] The data access unit reads the weight matrix data in discrete or continuous representation required by the operation module from the external address space;

[0375] Configure various discrete or continuous constants required for the forward operation of the neural network layer;

[0376] The master operation module first sends the input neuron vector to each slave operation module through the interconnection module, and stores it in the neuron cache unit that supports discrete data representation of the slave operation module;

[0377] The operation unit of the slave operation module reads the weight vector from the weight cache unit and the input neuron vector from the neuron cache unit of the slave operation module. If there is no discrete data representation in the vector, the dot product operation of the weight vector and the input neuron vector is completed. If there is discrete data representation in the vector, the discrete data operation module determines the corresponding bit operation according to the value of the discrete data instead of the dot product operation, and returns the obtained neuron value through the interconnection module;

[0378] In the interconnected module, the neuron values ​​returned from the operation modules are pieced together step by step to form a complete intermediate result vector;

[0379] The main operation module reads the bias vector of discrete representation or continuous representation from the neuron cache unit of the main operation module, adds it to the intermediate result vector returned by the interconnection module, and then activates the addition result to obtain the output neuron vector and writes it back to the neuron cache unit of the main operation module; and

[0380] The data access unit stores the output neuron vector in the neuron cache unit of the main operation module to a specified address in the external address space.

[0381] A method for performing a batch normalization operation using a forward operation apparatus:

[0382] The data access unit reads all artificial neural network operation instructions related to the batch of normalized forward operations from the external address space and caches them in the instruction cache unit;

[0383] The continuous-discrete conversion module reads the continuous data that needs to be converted by the neural network layer from the external address space, converts it into discrete data, and then stores it back into the external address space.

[0384] The data access unit reads all discrete or continuous data related to the batch normalization forward operation of the layer required by the main operation module from the external address space to the neuron cache unit of the main operation module;

[0385] Configure the various discrete or continuous constants required for the batch normalization forward operation of this layer;

[0386] The master operation module first sends the input neuron vector to each slave operation module through the interconnection module, and stores it in the neuron cache unit that supports discrete data representation of the slave operation module;

[0387] The operation unit of the slave operation module reads the weight vector from the weight cache unit, reads the input neuron vector from the neuron cache unit of the slave operation module, calculates the mean and standard deviation of the input vector at each batch scale, and returns the obtained neuron value through the interconnection module;

[0388] In the interconnected module, the neuron values ​​returned from the operation modules are pieced together step by step to form a complete intermediate result vector;

[0389] The main operation module reads the discrete or continuous representation input neuron vector from the neuron cache unit of the main operation module, subtracts it from the mean result vector returned by the interconnection module, and then divides the subtraction result and the standard deviation result to obtain the output neuron vector and write it back to the neuron cache unit of the main operation module;

[0390] The data access unit stores the output neuron vector in the neuron cache unit of the main operation module to a specified address in the external address space.

[0391] A method for performing a forward operation of a multi-layer artificial neural network, comprising:

[0392] For each layer, a method based on a single-layer artificial neural network forward operation or a batch normalization operation is performed, where:

[0393] After the execution of the previous layer of artificial neural network is completed, the output neuron address of the previous layer stored in the main operation module is used as the input neuron address of the current layer, and the single-layer artificial neural network forward operation method or the batch normalization operation method is executed again for the current layer.

[0394] A neural network computing device is provided, comprising:

[0395] A computing module for performing neural network operations; and

[0396] The power conversion module is connected to the operation module and is used to convert the input neuron data and / or output neuron data of the neural network operation into power neuron data.

[0397] In some embodiments, the power conversion module includes:

[0398] A first power conversion unit, configured to convert the neuron data output by the operation module into power neuron data; and

[0399] The second power conversion unit is used to convert the neuron data input into the operation module into power neuron data.

[0400] In some embodiments, the operation module further includes: a third power conversion unit, configured to convert power neuron data into non-power neuron data.

[0401] In some embodiments, the neural network computing device further includes:

[0402] Storage module, used for storing data and operation instructions;

[0403] The control module is used to control the interaction between data and operation instructions, which receives the data and operation instructions sent by the storage module and decodes the operation instructions into operation microinstructions; wherein,

[0404] The operation module includes an operation unit, which is used to receive the data and operation microinstructions sent by the control module, and perform neural network operations on the weight data and neuron data received according to the operation microinstructions.

[0405] In some embodiments, the control module includes: an operation instruction cache unit, a decoding unit, an input neuron cache unit, a weight cache unit and a data control unit; wherein

[0406] an operation instruction cache unit, connected to the data control unit, and configured to receive operation instructions sent by the data control unit;

[0407] A decoding unit connected to the operation instruction cache unit, used to read the operation instruction from the operation instruction cache unit and decode it into operation microinstructions;

[0408] an input neuron buffer unit connected to the data control unit and configured to obtain corresponding power neuron data from the data control unit;

[0409] A weight cache unit, connected to the data control unit, and configured to obtain corresponding weight data from the data control unit;

[0410] A data control unit is connected to the storage module and is used to implement data and operation instruction interaction between the storage module and the operation instruction cache unit, the weight cache unit and the input neuron cache unit respectively; wherein,

[0411] The operation unit is connected to the decoding unit, the input neuron cache unit and the weight cache unit respectively, receives operation microinstructions, power neuron data and weight data, and performs corresponding neural network operations on the power neuron data and weight data received by the operation unit according to the operation microinstructions.

[0412] In some embodiments, the neural network operation device further includes: an output module including an output neuron buffer unit for receiving neuron data output by the operation module;

[0413] Wherein, the power conversion module includes:

[0414] a first power conversion unit, connected to the output neuron buffer unit, for converting the neuron data output by the output neuron buffer unit into power neuron data;

[0415] a second power conversion unit, connected to the storage module, for converting the neuron data input into the storage module into power neuron data;

[0416] The operation module further includes: a third power conversion unit connected to the operation unit, and configured to convert the power neuron data into non-power neuron data.

[0417] In some embodiments, the first power conversion unit is also connected to the data control unit, and is used to convert the neuron data output by the operation module into power neuron data and send it to the data control unit as input data for the next layer of neural network operation.

[0418] In some embodiments, the power neuron data includes a sign bit and a power bit, the sign bit is used to represent the sign of the power neuron data, and the power bit is used to represent the power bit data of the power neuron data; the sign bit includes one or more bits of bit data, and the power bit includes m bits of bit data, where m is a positive integer greater than 1.

[0419] In some embodiments, the neural network operation device also includes a storage module, which pre-stores a coding table, which includes power bit data and exponential values. The coding table is used to obtain the exponential value corresponding to the power bit data through each power bit data of the power neuron data.

[0420] In some embodiments, the coding table further includes one or more power bit data set to zero, and the power neuron data corresponding to the power bit data set to zero is 0.

[0421] In some embodiments, the maximum power bit data corresponds to the power neuron data of 0 or the minimum power bit data corresponds to the power neuron data of 0.

[0422] In some embodiments, the corresponding relationship in the coding table is that the highest bit of the power bit data represents a zero bit, and the other m-1 bits of the power bit data correspond to the exponent value.

[0423] In some embodiments, the corresponding relationship of the coding table is a positive correlation relationship, the storage module pre-stores an integer value x and a positive integer value y, and the exponent value corresponding to the minimum power bit data is x; wherein x represents the bias value and y represents the step size.

[0424] In some embodiments, the exponential value corresponding to the smallest power bit data is x, the exponential value corresponding to the largest power bit data is 0, and the exponential value corresponding to other power bit data other than the smallest and largest power bit data is (power bit data + x) * y.

[0425] In some embodiments, y=1, and the value of x=-2m-1.

[0426] In some embodiments, the corresponding relationship of the coding table is a negative correlation relationship, the storage module pre-stores an integer value x and a positive integer value y, and the exponent value corresponding to the largest power bit data is x; wherein x represents the bias value and y represents the step size.

[0427] In some embodiments, the exponential value corresponding to the maximum power bit data is x, the exponential value corresponding to the minimum power bit data is 0, and the exponential value corresponding to other power bit data other than the minimum and maximum power bit data is (power bit data-x)*y.

[0428] In some embodiments, y=1, and the value of x is equal to 2m-1.

[0429] In some embodiments, converting the neuron data into power neuron data includes:

[0430] s out =s in

[0431]

[0432] Among them, d in is the input data of the power conversion unit, d out is the output data of the power conversion unit, s in is the symbol of the input data, s out is the symbol of the output data, d in+ is the positive part of the input data, d in+ =d in ×s in , d out+ is the positive part of the output data, d out+ =d out ×s out , Indicates that the data x is removed; or,

[0433] s out =s in

[0434]

[0435] Among them, d in is the input data of the power conversion unit, d out is the output data of the power conversion unit, s in is the symbol of the input data, s out is the symbol of the output data, d in+ is the positive part of the input data, d in+ =d in ×s in , dout+ is the positive part of the output data, d out+ =d out ×s out , |x| means to perform an upper integer operation on the data x; or,

[0436] s out =s in

[0437] d out+ =[log2(d in+ )]

[0438] Among them, d in is the input data of the power conversion unit, d out is the output data of the power conversion unit; s in is the symbol of the input data, s out is the symbol of the output data; d in+ is the positive part of the input data, d in+ =d in ×s in , d out+ is the positive part of the output data, d out+ =d out ×s out ; Indicates rounding operation on data x.

[0439] According to another aspect of the present disclosure, a neural network operation method is provided, comprising:

[0440] Perform neural network operations; and

[0441] Before executing the neural network operation, the input neuron data of the neural network operation is converted into power neuron data; and / or after executing the neural network operation, the output neuron data of the neural network operation is converted into power neuron data.

[0442] In some embodiments, before performing a neural network operation, the step of converting input neuron data of the neural network operation into power neuron data includes:

[0443] Converting non-power neuron data in the input data into power neuron data; and

[0444] Receive and store operation instructions, the power neuron data and weight data.

[0445] In some embodiments, between the steps of receiving and storing the operation instruction, the power neuron data and the weight data and the step of performing the neural network operation, the method further includes:

[0446] Read the operation instructions and decode them into various operation microinstructions.

[0447] In some embodiments, in the step of performing neural network operations, neural network operations are performed on weight data and power neuron data according to operation microinstructions.

[0448] In some embodiments, after performing a neural network operation, converting output neuron data of the neural network operation into power neuron data includes:

[0449] Output the neuron data obtained after the neural network operation; and

[0450] The non-power neuron data in the neuron data obtained after the neural network operation is converted into power neuron data.

[0451] In some embodiments, the non-power neuron data in the neuron data obtained after the neural network operation is converted into power neuron data and sent to the data control unit to serve as the input power neuron of the next layer of the neural network operation. The neural network operation steps and the steps of converting the non-power neuron data into power neuron data are repeated until the operation of the last layer of the neural network is completed.

[0452] In some embodiments, an integer value x and a positive integer value y are pre-stored in a storage module; wherein x represents a bias value and y represents a step size; by changing the integer value x and the positive integer value y pre-stored in the storage module, the range of power neuron data that can be represented by the neural network operation device is adjusted.

[0453] According to another aspect of the present disclosure, a method for using the neural network operation device is provided, wherein the range of power neuron data that can be represented by the neural network operation device is adjusted by changing the integer value x and the positive integer value y pre-stored in the storage module.

[0454] According to one aspect of the present disclosure, a neural network computing device is provided, comprising:

[0455] A computing module for performing neural network operations; and

[0456] The power conversion module is connected to the operation module and is used to convert the input data and / or output data of the neural network operation into power data.

[0457] In some embodiments, the input data includes input neuron data and input weight data, the output data includes output neuron data and output weight data, and the power data includes power neuron data and power weight data.

[0458] In some embodiments, the power conversion module includes:

[0459] A first power conversion unit, configured to convert the output data of the operation module into power data; and

[0460] The second power conversion unit is used to convert the input data of the operation module into power data.

[0461] In some embodiments, the operation module further includes: a third power conversion unit, configured to convert power data into non-power data.

[0462] In some embodiments, the neural network computing device further includes:

[0463] Storage module, used for storing data and operation instructions;

[0464] The control module is used to control the interaction between data and operation instructions, which receives the data and operation instructions sent by the storage module and decodes the operation instructions into operation microinstructions; wherein,

[0465] The operation module includes an operation unit, which is used to receive the data and operation microinstructions sent by the control module, and perform neural network operations on the weight data and neuron data received according to the operation microinstructions.

[0466] In some embodiments, the control module includes: an operation instruction cache unit, a decoding unit, an input neuron cache unit, a weight cache unit and a data control unit; wherein

[0467] an operation instruction cache unit, connected to the data control unit, and configured to receive operation instructions sent by the data control unit;

[0468] A decoding unit connected to the operation instruction cache unit, used to read the operation instruction from the operation instruction cache unit and decode it into operation microinstructions;

[0469] an input neuron buffer unit connected to the data control unit and configured to obtain corresponding power neuron data from the data control unit;

[0470] A weight cache unit, connected to the data control unit, for obtaining corresponding power weight data from the data control unit;

[0471] A data control unit is connected to the storage module and is used to implement data and operation instruction interaction between the storage module and the operation instruction cache unit, the weight cache unit and the input neuron cache unit respectively; wherein,

[0472] The operation unit is connected to the decoding unit, the input neuron cache unit and the weight cache unit respectively, receives operation microinstructions, power neuron data, and power weight data, and performs corresponding neural network operations on the power neuron data and power weight data received by the operation unit according to the operation microinstructions.

[0473] In some embodiments, the neural network operation device further includes: an output module including an output neuron buffer unit for receiving neuron data output by the operation module;

[0474] Wherein, the power conversion module includes:

[0475] a first power conversion unit, connected to the output neuron buffer unit and the operation unit, configured to convert the neuron data output by the output neuron buffer unit into power neuron data and convert the weight data output by the operation unit into power weight data;

[0476] A second power conversion unit, connected to the storage module, is used to convert the neuron data and weight data input into the storage module into power neuron data and power weight data respectively;

[0477] The operation module further includes: a third power conversion unit connected to the operation unit, for converting the power neuron data and the power weight data into non-power neuron data and non-power weight data respectively.

[0478] In some embodiments, the first power conversion unit is also connected to the data control unit, and is used to convert the neuron data and weight data output by the operation module into power neuron data and power weight data respectively and send them to the data control unit as input data for the next layer of neural network operation.

[0479] In some embodiments, the power neuron data includes a sign bit and a power bit, the sign bit is used to represent the sign of the power neuron data, and the power bit is used to represent the power bit data of the power neuron data; the sign bit includes one or more bit data, and the power bit includes m bit data, where m is a positive integer greater than 1;

[0480] The power weight data represents that the numerical value of the weight data is expressed in the form of its power exponential value, wherein the power weight data includes a sign bit and a power bit, the sign bit uses one or more bits to represent the sign of the weight data, and the power bit uses m bits to represent the power bit data of the weight data, where m is a positive integer greater than 1.

[0481] In some embodiments, the neural network operation device also includes a storage module, which pre-stores a coding table, which includes power bit data and exponential values. The coding table is used to obtain the exponential value corresponding to the power bit data through each power bit data of the power neuron data and the power weight data.

[0482] In some embodiments, the coding table further includes one or more power bit data set to zero, and the power neuron data and power weight data corresponding to the power bit data set to zero are 0.

[0483] In some embodiments, the maximum power bit data corresponds to power neuron data and the power weight data is 0, or the minimum power bit data corresponds to power neuron data and the power weight data is 0.

[0484] In some embodiments, the corresponding relationship in the coding table is that the highest bit of the power bit data represents a zero bit, and the other m-1 bits of the power bit data correspond to the exponent value.

[0485] In some embodiments, the corresponding relationship of the coding table is a positive correlation relationship, the storage module pre-stores an integer value x and a positive integer value y, and the exponent value corresponding to the minimum power bit data is x; wherein x represents the bias value and y represents the step size.

[0486] In some embodiments, the exponential value corresponding to the smallest power bit data is x, the exponential value corresponding to the largest power bit data is 0, and the exponential value corresponding to the other power bit data other than the smallest and largest power bit data is (power bit data + x) * y.

[0487] In some embodiments, y=1, the value of x=-2 m-1 .

[0488] In some embodiments, the corresponding relationship of the coding table is a negative correlation relationship, the storage module pre-stores an integer value x and a positive integer value y, and the exponent value corresponding to the largest power bit data is x; wherein x represents the bias value and y represents the step size.

[0489] In some embodiments, the exponential value corresponding to the largest power bit data is x, the power neuron data and power weight data corresponding to the smallest power bit data are 0, and the exponential value corresponding to other power bit data other than the smallest and largest power bit data is (power bit data-x)*y.

[0490] In some embodiments, y=1, and the value of x is equal to 2 m-1 .

[0491] In some embodiments, converting the neuron data and the weight data into power neuron data and power weight data, respectively, includes:

[0492] s out =s in

[0493]

[0494] Among them, d in is the input data of the power conversion unit, d out is the output data of the power conversion unit, s in is the symbol of the input data, s out is the symbol of the output data, d in+ is the positive part of the input data, d in+ =d in ×s in , d out+ is the positive part of the output data, d out+ =d out ×s out , Indicates that the data x is removed; or,

[0495] s out =s in

[0496]

[0497] Among them, d in is the input data of the power conversion unit, d out is the output data of the power conversion unit, s in is the symbol of the input data, s out is the symbol of the output data, d in+ is the positive part of the input data, d in+ =d in ×s in , d out+ is the positive part of the output data, d out+ =d out ×s out , Indicates that the data x is rounded up; or,

[0498] s out =s in

[0499] d out+ =[log2(d in+ )]

[0500] Among them, d in is the input data of the power conversion unit, d out is the output data of the power conversion unit; s in is the symbol of the input data, sout is the symbol of the output data; d in+ is the positive part of the input data, d in+ =d in ×s in , d out+ is the positive part of the output data, d out+ =d out ×s out ; Indicates rounding operation on data x.

[0501] According to another aspect of the present disclosure, a neural network operation method is provided, comprising:

[0502] Perform neural network operations; and

[0503] Before executing the neural network operation, the input data of the neural network operation is converted into power data; and / or after executing the neural network operation, the output data of the neural network operation is converted into power data.

[0504] In some embodiments, the input data includes input neuron data and input weight data, the output data includes output neuron data and output weight data, and the power data includes power neuron data and power weight data.

[0505] In some embodiments, before performing the neural network operation, the step of converting the input data of the neural network operation into power data includes:

[0506] Converting non-power data in the input data into power data; and

[0507] Receive and store the operation instruction and the power data.

[0508] In some embodiments, between the steps of receiving and storing the operation instruction and the power data and the step of performing the neural network operation, the method further includes:

[0509] Read the operation instructions and decode them into various operation microinstructions.

[0510] In some embodiments, in the step of performing neural network operations, neural network operations are performed on the power weight data and the power neuron data according to operation microinstructions.

[0511] In some embodiments, after performing a neural network operation, converting output data of the neural network operation into power data includes:

[0512] Output the data obtained after the neural network operation; and

[0513] Convert the non-power data in the data obtained after the neural network operation into power data.

[0514] In some embodiments, the non-power data in the data obtained after the neural network operation is converted into power data and sent to the data control unit to serve as input data for the next layer of the neural network operation. The neural network operation steps and the steps of converting the non-power data into power data are repeated until the last layer of the neural network operation is completed.

[0515] In some embodiments, an integer value x and a positive integer value y are pre-stored in a storage module; wherein x represents a bias value and y represents a step size; by changing the integer value x and the positive integer value y pre-stored in the storage module, the power data range that can be represented by the neural network computing device is adjusted.

[0516] According to another aspect of the present disclosure, a method for using the neural network operation device is provided, wherein the range of power data that can be represented by the neural network operation device is adjusted by changing the integer value x and the positive integer value y pre-stored in the storage module.

[0517] According to one aspect of the present disclosure, there is provided a computing device, comprising:

[0518] An operation control module, used for determining block information;

[0519] The operation module is used to perform block division, transposition and merging operations on the operation matrix according to the block information to obtain a transposed matrix of the operation matrix.

[0520] In some embodiments, the computing device further includes:

[0521] An address storage module, configured to store address information of the operation matrix; and

[0522] A data storage module, used for storing the operation matrix and the transposed matrix after the operation;

[0523] Among them, the operation control module is used to extract the address information of the operation matrix from the address storage module, and obtain block information based on the address information of the operation matrix; the operation module is used to obtain the address information and block information of the operation matrix from the operation control module, extract the operation matrix from the data storage module based on the address information of the operation matrix, and perform block, transposition and merge operations on the operation matrix based on the block information to obtain the transposed matrix of the operation matrix, and feed back the transposed matrix of the operation matrix to the data storage module.

[0524] In some embodiments, the operation module includes a matrix blocking unit, a matrix operation unit, and a matrix merging unit, wherein:

[0525] Matrix blocking unit: used to obtain the address information and blocking information of the operation matrix from the operation control module, extract the operation matrix from the data storage module according to the address information of the operation matrix, and block the operation matrix according to the blocking information to obtain n block matrices;

[0526] a matrix operation unit, configured to obtain the n block matrices and perform a transpose operation on the n block matrices to obtain a transposed matrix of the n block matrices;

[0527] A matrix merging unit is used to obtain and merge the transposed matrices of the n block matrices to obtain the transposed matrix of the operation matrix, and feed the transposed matrix of the operation matrix back to the data storage module, wherein n is a natural number.

[0528] In some embodiments, the operation module further includes a cache unit for caching the n block matrices for acquisition by the matrix operation unit.

[0529] In some embodiments, the operation control module includes an instruction processing unit, an instruction cache unit, and a matrix judgment unit, wherein:

[0530] An instruction cache unit, used to store matrix operation instructions to be executed;

[0531] An instruction processing unit, configured to obtain a matrix operation instruction from an instruction cache unit, decode the matrix operation instruction, and obtain address information of an operation matrix from the address storage module according to the decoded matrix operation instruction;

[0532] The matrix judgment unit is used to analyze the address information of the operation matrix to obtain the block information.

[0533] In some embodiments, the operation control module also includes a dependency processing unit for determining whether the decoded matrix operation instruction and the address information of the operation matrix conflict with the previous operation; if there is a conflict, the decoded matrix operation instruction and the address information of the operation matrix are temporarily stored; if there is no conflict, the decoded matrix operation instruction and the address information of the operation matrix are transmitted to the matrix judgment unit.

[0534] In some embodiments, the operation control module further includes an instruction queue memory for caching the decoded matrix operation instructions and address information of the operation matrix that have conflicts. When the conflict is eliminated, the cached decoded matrix operation instructions and address information of the operation matrix are transmitted to the matrix judgment unit.

[0535] In some embodiments, the instruction processing unit includes an instruction fetch unit and a decode unit, wherein:

[0536] an instruction fetch unit, configured to fetch a matrix operation instruction from the instruction cache unit and transmit the matrix operation instruction to the decoding unit;

[0537] A decoding unit is used to decode the matrix operation instruction, extract the address information of the operation matrix from the address storage module according to the decoded matrix operation instruction, and transmit the decoded matrix operation instruction and the extracted address information of the operation matrix to the dependency processing unit.

[0538] In some embodiments, the device further includes an input / output module for inputting the operation matrix data into the data storage module, and for obtaining the transposed matrix after operation from the data storage module, and outputting the transposed matrix after operation.

[0539] In some embodiments, the address storage module includes a scalar register file or a general memory unit; the data storage module includes a high-speed cache memory or a general memory unit; the address information of the operation matrix is ​​the starting address information and matrix size information of the matrix.

[0540] According to another aspect of the present disclosure, a calculation method is provided, comprising the following steps:

[0541] The operation control module determines the block information;

[0542] The operation module performs block division, transposition and merging operations on the operation matrix according to the block division information to obtain a transposed matrix of the operation matrix.

[0543] In some embodiments, the step of determining the block information by the operation control module includes:

[0544] The operation control module extracts address information of the operation matrix from the address storage module; and

[0545] The operation control module obtains the block information according to the address information of the operation matrix.

[0546] In some embodiments, the step of the operation control module extracting the address information of the operation matrix from the address storage module includes:

[0547] The instruction fetch unit extracts the operation instruction and sends the operation instruction to the decoding unit;

[0548] The decoding unit decodes the operation instruction, obtains the address information of the operation matrix from the address storage module according to the decoded operation instruction, and sends the decoded operation instruction and the address information of the operation matrix to the dependency processing unit;

[0549] The dependency processing unit analyzes whether the decoded operation instruction has a data dependency with the previous instruction that has not yet been executed. If so, the decoded operation instruction and the address information of the corresponding operation matrix need to wait in the instruction queue memory until the data dependency no longer exists between the decoded operation instruction and the previous instruction that has not yet been executed.

[0550] In some embodiments, the operation module performs block division, transposition, and merging operations on the operation matrix according to the block division information to obtain a transposed matrix of the operation matrix, including:

[0551] The operation module extracts the operation matrix from the data storage module according to the address information of the operation matrix; and divides the operation matrix into n block matrices according to the block information;

[0552] The operation module performs a transposition operation on each of the n block matrices to obtain a transposed matrix of the n block matrices; and

[0553] The operation module combines the transposed matrices of the n block matrices to obtain the transposed matrix of the operation matrix and feeds it back to the data storage module;

[0554] Wherein, n is a natural number.

[0555] In some embodiments, the operation module combines the transposed matrices of the n block matrices to obtain the transposed matrix of the operation matrix, and feeds the transposed matrix back to the data storage module, including:

[0556] The matrix merging unit receives the transposed matrix of each block matrix. When the number of received transposed matrices of the block matrices reaches the total number of blocks, the matrix merging unit performs a matrix merging operation on all blocks to obtain the transposed matrix of the operation matrix; and feeds the transposed matrix back to the specified address of the data storage module.

[0557] The input-output module directly accesses the data storage module and reads the transposed matrix of the operation matrix obtained by operation from the data storage module.

[0558] According to one aspect of the present disclosure, there is provided a data screening device, comprising:

[0559] a storage unit for storing data;

[0560] A register unit, used to store the data address in the storage unit;

[0561] The data screening module obtains the data address in the register unit, obtains the corresponding data in the storage unit according to the data address, and performs a screening operation on the obtained data to obtain a data screening result.

[0562] In some embodiments, the data screening module includes a data screening unit that performs a screening operation on the acquired data.

[0563] In some embodiments, the data screening module further includes: an I / O unit, an input data cache unit, and an output data cache unit;

[0564] The I / O unit moves the data stored in the storage unit to the input data cache unit;

[0565] The input data cache unit stores the data transported by the I / O unit;

[0566] The data screening unit takes the data transmitted from the input data buffer unit as input data, performs a screening operation on the input data, and transmits the output data to the output data buffer unit;

[0567] The output data cache unit stores output data.

[0568] In some embodiments, the input data includes data to be filtered and location information data, and the output data includes filtered data, or filtered data and related information thereof.

[0569] In some embodiments, the data to be screened is a vector or an array, and the position information data is a binary code, a vector or an array; the relevant information includes the vector length, the array size, and the space occupied.

[0570] In some embodiments, the data filtering unit scans each component of the location information data. If the component is 0, the data component to be filtered corresponding to the component is deleted. If the component is 1, the data component to be filtered corresponding to the component is retained; or, if the component is 1, the data component to be filtered corresponding to the component is deleted. If the component is 0, the data component to be filtered corresponding to the component is retained. After the scanning is completed, the filtered data is obtained and output.

[0571] In some embodiments, the data screening module further includes a structure deformation unit for deforming the storage structure of the input data and / or output data.

[0572] According to another aspect of the present disclosure, a data screening method is provided, wherein data screening is performed using the data screening device, comprising:

[0573] Step A: The data filtering module obtains the data address in the register unit;

[0574] Step B: obtaining corresponding data in the storage unit according to the data address; and

[0575] Step C: Perform a screening operation on the acquired data to obtain data screening results.

[0576] In some embodiments, the step A includes: the data screening unit obtains the address of the data to be screened and the address of the location information data from the register unit;

[0577] The step B comprises:

[0578] Sub-step B1: the I / O unit transmits the data to be filtered and the location information data in the storage unit to the input data cache unit;

[0579] Sub-step B2: the input data cache unit transmits the data to be screened and the location information data to the data screening unit;

[0580] The step C includes: the data screening unit performs a screening operation on the data to be screened according to the position information data, and transmits the output data to the output data cache unit.

[0581] In some embodiments, between sub-step B1 and sub-step B2, the following steps are further included:

[0582] Determine whether to perform storage structure deformation. If yes, execute sub-step B3; if not, directly execute sub-step B2;

[0583] Sub-step B3: the input data cache unit transmits the data to be screened to the structural deformation unit, the structural deformation unit performs storage structural deformation, transmits the deformed data to be screened back to the input data cache unit, and then executes sub-step B2.

[0584] According to one aspect of the present disclosure, a neural network processor is provided, comprising: a memory, a high-speed scratchpad memory, and a heterogeneous core; wherein,

[0585] The memory is used to store data and instructions for neural network operations;

[0586] The high-speed temporary storage memory is connected to the memory via a memory bus;

[0587] The heterogeneous core is connected to the high-speed temporary memory through a high-speed temporary memory bus, reads data and instructions of neural network operations through the high-speed temporary memory, completes the neural network operations, and sends the operation results back to the high-speed temporary memory, and controls the high-speed temporary memory to write the operation results back to the memory.

[0588] In some embodiments, the heterogeneous core includes:

[0589] a plurality of computing cores having at least two different types of computing cores for performing neural network operations or neural network layer operations; and

[0590] One or more logic control cores are used to determine whether the dedicated core and / or the general core performs the neural network operation or the neural network layer operation based on the data of the neural network operation.

[0591] In some embodiments, the multiple computing cores include x general-purpose cores and y dedicated cores; wherein the dedicated cores are dedicated to executing specified neural network / neural network layer operations, and the general-purpose cores are used to execute arbitrary neural network / neural network layer operations.

[0592] In some embodiments, the general-purpose core is a CPU and the specialized core is an NPU.

[0593] In some embodiments, the high-speed scratch pad memory includes a shared high-speed scratch pad memory and / or a non-shared high-speed scratch pad memory; wherein the shared high-speed scratch pad memory is connected to at least two cores of the heterogeneous cores via a high-speed scratch pad memory bus; and the non-shared high-speed scratch pad memory is connected to a core of the heterogeneous core via a high-speed scratch pad memory bus.

[0594] In some embodiments, the logic control core is connected to the high-speed cache memory through a high-speed cache memory bus, reads the data of the neural network operation through the high-speed cache memory, and decides whether to use a dedicated core and / or a general core as the target core to perform the neural network operation and / or neural network layer operation based on the type and parameters of the neural network model in the data of the neural network operation.

[0595] In some embodiments, the logic control core sends a signal directly to the target core through a control bus or sends a signal to the target core through the high-speed cache memory, thereby controlling the target core to perform neural network operations and / or neural network layer operations.

[0596] According to another aspect of the present disclosure, a neural network operation method is provided, wherein the neural network operation is performed using the neural network processor, comprising:

[0597] The logic control core in the heterogeneous core reads the data and instructions for the neural network operation from the memory through the high-speed scratchpad memory; and

[0598] The logic control core in the heterogeneous core determines whether a dedicated core and / or a general core performs the neural network operation or the neural network layer operation based on the type and parameters of the neural network model in the data of the neural network operation.

[0599] In some embodiments, the logic control core in the heterogeneous core determines whether to execute the neural network layer operation by the dedicated core and / or the general core based on the type and parameters of the neural network model in the data of the neural network operation, including:

[0600] The logic control core in the heterogeneous core determines whether there is a qualified dedicated core based on the type and parameters of the neural network model in the data of the neural network operation;

[0601] If the dedicated core m meets the conditions, the dedicated core m is used as the target core, and the logic control core in the heterogeneous core sends a signal to the target core to send the data and address corresponding to the neural network operation instruction to the target core;

[0602] The target kernel obtains data and instructions for the neural network operation from the memory through the shared or non-shared high-speed temporary memory according to the address, performs the neural network operation, and outputs the operation result to the memory through the shared or non-shared high-speed temporary memory, and the operation is completed;

[0603] If there is no dedicated core that meets the requirements, the logic control core in the heterogeneous core sends a signal to the general core to send the data and address corresponding to the neural network operation instruction to the general core;

[0604] The general core obtains the data and instructions of the neural network operation from the memory through the shared or non-shared high-speed cache memory according to the address, performs the neural network operation, and outputs the operation result to the memory through the shared or non-shared high-speed cache memory, and the operation is completed.

[0605] In some embodiments, the qualified dedicated core refers to a dedicated core that supports specified neural network operations and can complete specified neural network operation scales.

[0606] In some embodiments, the logic control core in the heterogeneous core determines whether to execute the neural network operation by the dedicated core and / or the general core based on the type and parameters of the neural network model in the neural network operation data, including:

[0607] The logic control core in the heterogeneous core analyzes the type and parameters of the neural network model in the data, determines whether there is a qualified dedicated core for each neural network layer, and allocates a corresponding general core or dedicated core to each neural network layer to obtain a kernel sequence corresponding to the neural network layer;

[0608] The logic control core in the heterogeneous core sends the data and addresses corresponding to the instructions of the neural network layer operation to the dedicated core or general core corresponding to the neural network layer, and sends the number of the next dedicated core or general core in the core sequence to the dedicated core or general core corresponding to the neural network layer;

[0609] The dedicated core and the general core corresponding to the neural network layer read the data and instructions of the neural network layer operation from the address, perform the neural network layer operation, and transmit the operation results to the specified address of the shared and / or non-shared high-speed temporary storage memory;

[0610] The logic control core controls the shared and / or non-shared high-speed temporary memory to write the operation results of the neural network layer back to the memory, and the operation is completed.

[0611] In some embodiments, the qualified dedicated kernel refers to a dedicated kernel that supports the specified neural network layer operation and can complete the specified neural network layer operation scale.

[0612] In some embodiments, the neural network operation includes a pulse neural network operation; the neural network layer operation includes a convolution operation, a fully connected layer, a splicing operation, a bitwise addition / multiplication operation, a Relu operation, a pooling operation and / or a Batch Norm operation of the neural network layer.

[0613] A processing device, comprising:

[0614] The coarse-grained branch unit is used to perform coarse-grained pruning on the weights of the neural network to obtain the pruned weights;

[0615] A computing unit, configured to train the neural network according to the pruned weights;

[0616] The coarse-grained pruning unit is specifically used for:

[0617] M weights are selected from the weights of the neural network through a sliding window, where M is an integer greater than 1; when the M weights meet a preset condition, all or part of the M weights are set to zero.

[0618] Furthermore, the preset condition is that the amount of information of the M weights is less than a first preset threshold.

[0619] Furthermore, the information amount of the M weights is the arithmetic mean of the absolute values ​​of the M weights, the geometric mean of the absolute values ​​of the M weights, or the maximum value of the M weights, the first preset threshold is the first threshold, the second threshold, or the third threshold, and the information amount of the M weights is less than the first preset threshold, including:

[0620] The arithmetic mean of the absolute values ​​of the M weights is less than the first threshold, or the geometric mean of the absolute values ​​of the M weights is less than the second threshold, or the maximum value of the M weights is less than the third threshold.

[0621] Furthermore, the coarse-grained pruning unit is used to repeatedly perform coarse-grained pruning on the weights of the neural network and train the neural network based on the pruned weights until no weights meet the preset conditions without losing the set accuracy.

[0622] Furthermore, the setting accuracy is x%, where x is between 0 and 5.

[0623] Furthermore, the neural network includes a fully connected layer, a convolutional layer and / or a long short-term memory (LSTM) layer, wherein the weight of the fully connected layer is a two-dimensional matrix (Nin, Nout), where Nin is the number of input neurons, Nout is the number of output neurons, and the fully connected layer has Nin*Nout weights; the weight of the convolutional layer is a four-dimensional matrix (Nfin, Nfout, Kx, Ky), where Nfin is the number of input feature images, Nfout is the number of output feature images, (Kx, Ky) is the size of the convolution kernel. The convolution layer has Nfin*Nfout*Kx*Ky weights. The weight of the LSTM layer is composed of the weights of m fully connected layers, where m is an integer greater than 0. The weight of the i-th fully connected layer is (Nin_i, Nout_i), where i is an integer greater than 0 and less than or equal to m. Nin_i represents the number of input neurons of the i-th fully connected layer weight, and Nout_i represents the number of output neurons of the i-th fully connected layer weight. The coarse-grained pruning unit is specifically used to:

[0624] When a coarse-grained pruning operation is performed on the weights of the fully connected layer, the size of the sliding window is a sliding window of Bin*Bout, where Bin is an integer greater than 0 and less than or equal to Nin, and Bout is an integer greater than 0 and less than or equal to Nout;

[0625] The sliding window can slide along the direction of Bin according to the step size Sin, or along

[0626] Slide in the direction of Bout according to the step size Sout, where Sin is a positive integer greater than 0 and less than or equal to Bin, and Sout is a positive integer greater than 0 and less than or equal to Bout;

[0627] Selecting M values ​​from the Nin*Nout weights through the sliding window, and when the M weights meet the preset condition, setting all or part of the M weights to zero, wherein M=Bin*Bout;

[0628] When the coarse-grained pruning operation is performed on the weights of the convolutional layer, the sliding window is a four-dimensional sliding window of size Bfin*Bfout*Bx*By, where Bfin is an integer greater than 0 and less than or equal to Nfin, Bfout is an integer greater than 0 and less than or equal to Nfout, Bx is an integer greater than 0 and less than or equal to Kx, and By is an integer greater than 0 and less than or equal to Ky;

[0629] Slide the sliding window along the direction of Bfin with a step size Sfin, or slide along the direction of Bfout with a step size Sfout, or slide along the direction of Bx with a step size S, or slide along the direction of By with a step size Sy, where Sfin is an integer greater than 0 and less than or equal to Bfin, Sfout is an integer greater than 0 and less than or equal to Bfout, Sx is an integer greater than 0 and less than or equal to Bx, and Sy is an integer greater than 0 and less than or equal to By;

[0630] Selecting M weights from the Nfin*Nfout*Kx*Ky weights through the sliding window, and setting all or part of the M weights to zero when the M weights meet the preset condition, wherein M=Bfin*Bfout*Bx*By;

[0631] When the coarse-grained pruning operation is performed on the weights of the LSTM layer, the size of the sliding window is Bin_i*Bout_i, where Bin_i is an integer greater than 0 and less than or equal to Nin_i, and Bout_i is an integer greater than 0 and less than or equal to Nout_i;

[0632] Slide the sliding window along the direction of Bin_i according to the step size Sin_i, or slide along the direction of Bout_i according to the step size Sout_i, where Sin_i is a positive integer greater than 0 and less than or equal to Bin_i, and Sout_i is a positive integer greater than 0 and less than or equal to Bout_i;

[0633] M weights are selected from the Bin_i*Bout_i weights through the sliding window, and when the M weights meet the preset condition, all or part of the M weights are set to zero, wherein M=Bin_i*Bout_i.

[0634] Furthermore, the operation unit is specifically configured to perform retraining based on the pruned weights and by using a back propagation algorithm.

[0635] Furthermore, the processing group device also includes:

[0636] A quantization unit is used to quantize the weights of the neural network and / or perform a first operation on the weights of the neural network after coarse-grained pruning of the weights of the neural network and before retraining the neural network based on the pruned weights to reduce the number of bits of the weights of the neural network.

[0637] An acceleration device, comprising:

[0638] A storage unit for storing input neurons, output neurons, pruned neural network weights, and instructions of a neural network; wherein the neural network is a trained neural network model obtained by training the pruned weights;

[0639] a coarse-grained pruning unit, configured to perform coarse-grained pruning on the weights of the neural network to obtain pruned weights, and store the pruned weights in the storage unit;

[0640] a coarse-grained number selection unit, configured to receive input neuron and target weight position information and select neurons corresponding to the target weight, wherein the target weight is a weight whose absolute value is greater than a second preset threshold;

[0641] The operation unit is used to receive the input target weight and its corresponding neuron, perform operations based on the target weight and its corresponding neuron, and retransmit the output neuron to the storage unit.

[0642] The storage unit is also used to store intermediate results generated during the operation of the operation unit.

[0643] Furthermore, the acceleration device further comprises:

[0644] The instruction control unit is used to receive the instruction and decode the instruction to obtain control information to control the operation unit.

[0645] Furthermore, the storage unit is used to store the target weight and the location information of the target weight.

[0646] Furthermore, the acceleration device further comprises:

[0647] The preprocessing unit is used to preprocess the original data and input the preprocessed data into the storage part. The above original data includes input neurons, output neurons and weights.

[0648] Furthermore, the preprocessing includes data segmentation, Gaussian filtering, binarization, regularization and / or normalization.

[0649] Furthermore, the acceleration device further comprises:

[0650] An instruction cache unit is used to cache the instructions. The instruction cache unit is an on-chip cache.

[0651] Furthermore, the acceleration device further comprises:

[0652] The target weight value cache unit is used to cache the target weight value. The target weight value cache unit is an on-chip cache.

[0653] Furthermore, the acceleration device further comprises:

[0654] The target weight value location cache unit is used to cache the location information of the target weight value. The target weight value location cache unit is an on-chip cache.

[0655] Furthermore, the acceleration device further comprises:

[0656] The input neuron cache unit is used to cache input neurons, and the input neuron cache unit is an on-chip cache.

[0657] Furthermore, the acceleration device further comprises:

[0658] The output neuron cache unit is used to cache output neurons, and the output neuron cache unit is an on-chip cache.

[0659] Furthermore, the target weight position cache unit is used to cache the position information of the target weight; the target weight position cache unit maps each connection weight in the input data to a corresponding input neuron one by one.

[0660] Furthermore, the acceleration device further comprises:

[0661] A direct access unit DMA is used to read and write data or instructions in the storage unit, the instruction cache unit, the coarse-grained pruning unit, the target weight cache unit, the target weight position cache unit, the input neuron cache unit or the output neuron cache.

[0662] Furthermore, the operation unit includes at least one of the following: a multiplier, used to multiply the first input data and the second input data to obtain the multiplied data; an addition tree, used to add the third input data step by step through the addition tree, or to add the third input data and the fourth input data to obtain the added data; an activation function operation unit, which performs an activation function operation on the fifth data to obtain output data, and the activation function is a sigmoid, tanh, relu or softmax function operation.

[0663] Furthermore, the operation unit also includes a pooling unit, which is used to obtain output data after the pooling operation by performing a pooling operation on the input sixth data, and the pooling operation includes: average pooling, maximum pooling or median pooling.

[0664] An acceleration device, comprising:

[0665] A storage unit for storing input neurons, output neurons, pruned neural network weights, and instructions of a neural network; wherein the neural network is a trained neural network model obtained by training the pruned weights;

[0666] A coarse-grained pruning unit is used to prune the weights of the neural network to obtain pruned weights, and store the pruned weights in the above-mentioned storage unit;

[0667] An operation unit, used for training the neural network according to the pruned weights to obtain a trained neural network;

[0668] A coarse-grained number selection unit is used to receive input neuron and target weight position information, select the input neuron corresponding to the target weight, and the target weight is a weight whose absolute value is greater than a second preset threshold; wherein the target weight is the trained weight.

[0669] The operation unit is used to receive the input target weight and its corresponding input neuron, perform operations based on the target weight and its corresponding input neuron, and retransmit the output neuron to the storage unit.

[0670] The storage unit may also be used to store intermediate results generated during the operation of the operation unit.

[0671] Furthermore, the acceleration device further comprises:

[0672] The instruction control unit is used to receive the instruction and decode the instruction to obtain control information for controlling the operation unit.

[0673] Furthermore, the storage unit is used to store the target weight and the location information of the target weight.

[0674] Furthermore, the acceleration device further comprises:

[0675] The preprocessing unit is used to preprocess the original data and input the preprocessed data into the storage part. The above original data includes the input neurons, output neurons and weights of the trained neural network.

[0676] Furthermore, the preprocessing includes data segmentation, Gaussian filtering, binarization, regularization and / or normalization.

[0677] Furthermore, the acceleration device further comprises:

[0678] An instruction cache unit is used to cache the instructions. The instruction cache unit is an on-chip cache.

[0679] Furthermore, the acceleration device further comprises:

[0680] The target weight value cache unit is used to cache the target weight value. The target weight value cache unit is an on-chip cache.

[0681] Furthermore, the acceleration device further comprises:

[0682] The target weight value location cache unit is used to cache the location information of the target weight value. The target weight value location cache unit is an on-chip cache.

[0683] Furthermore, the acceleration device further comprises:

[0684] The input neuron cache unit is used to cache input neurons, and the input neuron cache unit is an on-chip cache.

[0685] Furthermore, the acceleration device further comprises:

[0686] The output neuron cache unit is used to cache output neurons, and the output neuron cache unit is an on-chip cache.

[0687] Furthermore, the target weight position cache unit is used to cache the position information of the target weight, and the target weight position cache unit maps each connection weight in the input data to a corresponding input neuron one by one.

[0688] Furthermore, the acceleration device further comprises:

[0689] A direct access unit DMA is used to read and write data or instructions in the storage unit, the instruction cache unit, the coarse-grained pruning unit, the target weight cache unit, the target weight position cache unit, the input neuron cache unit or the output neuron cache.

[0690] Furthermore, the operation unit includes at least one of the following: a multiplier, used to multiply the first input data and the second input data to obtain the multiplied data; an addition tree, used to add the third input data step by step through the addition tree, or to add the third input data and the fourth input data to obtain the added data; an activation function operation unit, which performs an activation function operation on the fifth data to obtain output data, and the activation function is a sigmoid, tanh, relu or softmax function operation.

[0691] Furthermore, the operation unit also includes a pooling unit, which is used to obtain output data after the pooling operation by performing a pooling operation on the input sixth data, and the pooling operation includes: average pooling, maximum pooling or median pooling.

[0692] A processing method comprising:

[0693] Perform coarse-grained pruning on the weights of the neural network to obtain pruned weights;

[0694] Training the neural network according to the pruned weights;

[0695] The coarse-grained pruning of the neural network to obtain pruned weights includes:

[0696] Selecting M weights from the weights of the neural network through a sliding window, where M is an integer greater than 1;

[0697] When the M weights meet a preset condition, all or part of the M weights are set to zero to obtain pruned weights.

[0698] Furthermore, the preset conditions are:

[0699] The amount of information of the M weights is less than a first preset threshold.

[0700] Furthermore, the information amount of the M weights is the arithmetic mean of the absolute values ​​of the M weights, the geometric mean of the absolute values ​​of the M weights, or the maximum value of the M weights, the first preset threshold is the first threshold, the second threshold, or the third threshold, and the information amount of the M weights is less than the first preset threshold, including:

[0701] The arithmetic mean of the absolute values ​​of the M weights is less than the first threshold, or the geometric mean of the absolute values ​​of the M weights is less than the second threshold, or the maximum value of the M weights is less than the third threshold.

[0702] Furthermore, the method further comprises:

[0703] Repeating coarse-grained pruning of the weights of the neural network and training based on the pruned weights until no weights meet the preset conditions without losing the set accuracy.

[0704] Furthermore, the setting accuracy is x%, where x is between 0 and 5.

[0705] Furthermore, the neural network includes a fully connected layer, a convolutional layer and / or a long short-term memory (LSTM) layer, wherein the weight of the fully connected layer is a two-dimensional matrix (Nin, Nout), wherein Nin is the number of input neurons, Nout is the number of output neurons, and the fully connected layer has Nin*Nout weights; the weight of the convolutional layer is a four-dimensional matrix (Nfin, Nfout, Kx, Ky), wherein Nfin is the number of input feature images, Nfout is the number of output feature images, (K x, Ky) is the size of the convolution kernel, the convolution layer has Nfin*Nfout*Kx*Ky weights; the weight of the LSTM layer is composed of the weights of m fully connected layers, m is an integer greater than 0, the weight of the i-th fully connected layer is (Nin_i, Nout_i), where i is an integer greater than 0 and less than or equal to m, Nin_i represents the number of input neurons of the i-th fully connected layer weight, and Nout_i represents the number of output neurons of the i-th fully connected layer weight; the coarse-grained pruning of the neural network includes:

[0706] When coarse-grained pruning is performed on the weights of the fully connected layer of the neural network, the size of the sliding window is a sliding window of Bin*Bout, where Bin is an integer greater than 0 and less than or equal to Nin, and Bout is an integer greater than 0 and less than or equal to Nout;

[0707] The sliding window is enabled to slide along the direction of Bin according to a step size Sin, or to slide along the direction of Bout according to a step size Sout, where Sin is a positive integer greater than 0 and less than or equal to Bin, and Sout is a positive integer greater than 0 and less than or equal to Bout;

[0708] M values ​​are selected from the Nin*Nout weights through the sliding window, and when the M weights meet the preset conditions, all or part of the M weights are set to zero, wherein M=Bin*Bout.

[0709] When coarse-grained pruning is performed on the weights of the convolutional layer of the neural network, the sliding window is a four-dimensional sliding window of size Bfin*Bfout*Bx*By, where Bfin is an integer greater than 0 and less than or equal to Nfin, Bfout is an integer greater than 0 and less than or equal to Nfout, Bx is an integer greater than 0 and less than or equal to Kx, and By is an integer greater than 0 and less than or equal to Ky;

[0710] Slide the sliding window along the direction of Bfin with a step size Sfin, or slide along the direction of Bfout with a step size Sfout, or slide along the direction of Bx with a step size S, or slide along the direction of By with a step size Sy, where Sfin is an integer greater than 0 and less than or equal to Bfin, Sfout is an integer greater than 0 and less than or equal to Bfout, Sx is an integer greater than 0 and less than or equal to Bx, and Sy is an integer greater than 0 and less than or equal to By;

[0711] M weights are selected from the Nfin*Nfout*Kx*Ky weights through the sliding window, and when the M weights meet the preset conditions, all or part of the M weights are set to zero, wherein M=Bfin*Bfout*Bx*By.

[0712] When coarse-grained pruning is performed on the weights of the LSTM layer of the neural network, the size of the sliding window is Bin_i*Bout_i, where Bin_i is an integer greater than 0 and less than or equal to Nin_i, and Bout_i is an integer greater than 0 and less than or equal to Nout_i. The coarse-grained pruning of the weights of the LSTM layer of the neural network specifically includes:

[0713] Slide the sliding window along the direction of Bin_i according to the step size Sin_i, or slide along the direction of Bout_i according to the step size Sout_i, where Sin_i is a positive integer greater than 0 and less than or equal to Bin_i, and Sout_i is a positive integer greater than 0 and less than or equal to Bout_i;

[0714] M weights are selected from the Bin_i*Bout_i weights through the sliding window.

[0715] When the M weights meet the preset condition, all or part of the M weights are set to zero, wherein M=Bin_i*Bout_i.

[0716] Furthermore, the training of the neural network according to the pruned weights is specifically as follows:

[0717] The neural network is retrained according to the pruned weights and through a back propagation algorithm.

[0718] Furthermore, between the coarse-grained pruning and retraining of the neural network, the following steps are further included:

[0719] The weights of the neural network are quantized and / or a first operation is performed on the weights of the neural network to reduce the number of weight bits.

[0720] A neural network computing device, comprising one or more accelerators as described above, configured to obtain data to be computed and control information from other processing devices, perform specified neural network operations, and transmit the results of the operations to other processing devices via an I / O interface;

[0721] When the neural network computing device includes a plurality of computing devices, the plurality of computing devices may be connected and data may be transmitted between the plurality of computing devices via a specific structure;

[0722] Among them, multiple computing devices are interconnected and transmit data through a fast peripheral device interconnection bus PCIE bus to support larger-scale neural network operations; multiple computing devices share the same control system or have their own control systems; multiple computing devices share memory or have their own memory; the interconnection method of multiple computing devices is any interconnection topology.

[0723] A neural network chip, comprising the above-mentioned processing device, acceleration device and / or neural network computing device.

[0724] A chip packaging structure includes the neural network chip described in the sixth aspect.

[0725] A board comprising the neural network chip described in the sixth aspect or the chip packaging structure described in the seventh aspect.

[0726] An electronic device comprises the board card described in the first aspect.

[0727] Furthermore, the electronic device includes a data processing device, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a mobile phone, a driving recorder, a navigator, a sensor, a camera, a cloud server, a camera, a camcorder, a projector, a watch, a headset, a mobile storage, a wearable device, a vehicle, a household appliance, and / or a medical device.

[0728] Furthermore, the means of transportation include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; and the medical equipment include magnetic resonance imaging (MRI), ultrasound machines and / or electrocardiographs.

[0729] A processing device includes a storage unit, a coarse-grained number selection unit and an operation unit.

[0730] Storage unit, used to store the input neurons, output neurons, weights and instructions of the neural network;

[0731] A coarse-grained pruning unit is configured to perform coarse-grained pruning on the weights of the neural network to obtain pruned weights, and store the pruned weights and target weight position information in the storage unit; the target weight is a weight whose absolute value is greater than a second preset threshold, wherein the coarse-grained pruning unit is specifically configured to:

[0732] Selecting M weights from the weights of the neural network through a sliding window, where M is an integer greater than 1;

[0733] When the M weights meet a preset condition, setting all or part of the M weights to zero;

[0734] A computing unit, configured to perform training based on the pruned weights; weights that have been set to zero during training remain zero;

[0735] a coarse-grained number selection unit, configured to receive input neurons and target weight position information, and select input neurons corresponding to the target weight according to the target weight position information;

[0736] The operation unit is further used to complete the neural network operation according to the input target weight and the input neuron corresponding to the target weight to obtain the output neuron, and transmit the output neuron as the input neuron of the next layer to the storage unit.

[0737] Furthermore, the preset conditions include:

[0738] The amount of information of the M weights is less than a first preset threshold.

[0739] Furthermore, the information amount of the M weights is the arithmetic mean of the absolute values ​​of the M weights, the geometric mean of the absolute values ​​of the M weights, or the maximum value of the M weights, the first preset threshold is the first threshold, the second threshold, or the third threshold, and the information amount of the M weights is less than the first preset threshold, including:

[0740] The arithmetic mean of the absolute values ​​of the M weights is less than the first threshold, or the geometric mean of the absolute values ​​of the M weights is less than the second threshold, or the maximum value of the M weights is less than the third threshold.

[0741] Furthermore, the coarse-grained pruning unit and the computing unit are used to:

[0742] Repeating coarse-grained pruning of the weights of the neural network and training based on the pruned weights until no weights meet the preset conditions without losing the set accuracy.

[0743] Furthermore, the neural network includes a fully connected layer, a convolutional layer and / or a long short-term memory (LSTM) layer, wherein the weight of the fully connected layer is a two-dimensional matrix (Nin, Nout), where Nin is the number of input neurons, Nout is the number of output neurons, and the fully connected layer has Nin*Nout weights; the weight of the convolutional layer is a four-dimensional matrix (Nfin, Nfout, Kx, Ky), where Nfin is the number of input feature images, Nfout is the number of output feature images, (Kx, Ky) is the size of the convolution kernel. The convolution layer has Nfin*Nfout*Kx*Ky weights. The weight of the LSTM layer is composed of the weights of m fully connected layers, where m is an integer greater than 0. The weight of the i-th fully connected layer is (Nin_i, Nout_i), where i is an integer greater than 0 and less than or equal to m. Nin_i represents the number of input neurons of the i-th fully connected layer weight, and Nout_i represents the number of output neurons of the i-th fully connected layer weight. The coarse-grained pruning unit is specifically used to:

[0744] When a coarse-grained pruning operation is performed on the weights of the fully connected layer, the size of the sliding window is a sliding window of Bin*Bout, where Bin is an integer greater than 0 and less than or equal to Nin, and Bout is an integer greater than 0 and less than or equal to Nout;

[0745] The sliding window can slide along the direction of Bin according to the step size Sin, or along

[0746] Slide in the direction of Bout according to the step size Sout, where Sin is a positive integer greater than 0 and less than or equal to Bin, and Sout is a positive integer greater than 0 and less than or equal to Bout;

[0747] Selecting M values ​​from the Nin*Nout weights through the sliding window, and when the M weights meet the preset condition, setting all or part of the M weights to zero, wherein M=Bin*Bout;

[0748] When the coarse-grained pruning operation is performed on the weights of the convolutional layer, the sliding window is a four-dimensional sliding window of size Bfin*Bfout*Bx*By, where Bfin is an integer greater than 0 and less than or equal to Nfin, Bfout is an integer greater than 0 and less than or equal to Nfout, Bx is an integer greater than 0 and less than or equal to Kx, and By is an integer greater than 0 and less than or equal to Ky;

[0749] Slide the sliding window along the direction of Bfin with a step size Sfin, or slide along the direction of Bfout with a step size Sfout, or slide along the direction of Bx with a step size S, or slide along the direction of By with a step size Sy, where Sfin is an integer greater than 0 and less than or equal to Bfin, Sfout is an integer greater than 0 and less than or equal to Bfout, Sx is an integer greater than 0 and less than or equal to Bx, and Sy is an integer greater than 0 and less than or equal to By;

[0750] Selecting M weights from the Nfin*Nfout*Kx*Ky weights through the sliding window, and setting all or part of the M weights to zero when the M weights meet the preset condition, wherein M=Bfin*Bfout*Bx*By;

[0751] When the coarse-grained pruning operation is performed on the weights of the LSTM layer, the size of the sliding window is Bin_i*Bout_i, where Bin_i is an integer greater than 0 and less than or equal to Nin_i, and Bout_i is an integer greater than 0 and less than or equal to Nout_i;

[0752] Slide the sliding window along the direction of Bin_i according to the step size Sin_i, or slide along the direction of Bout_i according to the step size Sout_i, where Sin_i is a positive integer greater than 0 and less than or equal to Bin_i, and Sout_i is a positive integer greater than 0 and less than or equal to Bout_i;

[0753] M weights are selected from the Bin_i*Bout_i weights through the sliding window, and when the M weights meet the preset condition, all or part of the M weights are set to zero, wherein M=Bin_i*Bout_i.

[0754] Furthermore, the processing device further includes: an instruction control unit, configured to receive the instruction and generate a control instruction to control the operation unit after decoding.

[0755] Furthermore, the weights stored in the storage unit are target weights and target weight position data.

[0756] Furthermore, the processing device also includes a preprocessing unit: used to preprocess the input neurons and weights, and input the preprocessed data into the storage unit.

[0757] Furthermore, the preprocessing includes data segmentation, Gaussian filtering, binarization, regularization and / or normalization.

[0758] Furthermore, the processing device further includes an instruction cache unit for caching the instructions.

[0759] Furthermore, the processing device further includes a target weight cache unit for caching target weight data.

[0760] Furthermore, the processing device further includes a target weight position cache unit for caching target weight position data.

[0761] Furthermore, the processing device also includes an input neuron cache unit for caching input neurons.

[0762] Furthermore, the processing device also includes an output neuron cache unit for caching output neurons.

[0763] Furthermore, the instruction cache unit, the target weight cache unit, the target weight position cache unit, the input neuron cache unit or the output neuron cache unit is an on-chip cache.

[0764] Furthermore, the target weight position cache unit is used to cache target weight position data; the target weight position cache maps each connection weight in the input data to a corresponding input neuron one by one.

[0765] Furthermore, the processing device also includes a direct memory access (DMA) unit for reading and writing data or instructions in the storage unit, the instruction cache unit, the target weight cache unit, the target weight position cache unit, the input neuron cache unit, or the output neuron cache unit.

[0766] Furthermore, the computing unit includes at least one of the following:

[0767] A multiplier, configured to multiply the first input data and the second input data to obtain multiplied data;

[0768] one or more adders for adding the third input data;

[0769] The activation function operation unit is used to obtain output data by performing an activation function operation on the fifth data, where the activation function includes a sigmoid, tanh, relu or softmax function.

[0770] Furthermore, the operation unit includes a plurality of adders, and the plurality of adders form an addition tree, which is used to add the third input data stage by stage through the addition tree.

[0771] Furthermore, the operation unit also includes a pooling unit for performing a pooling operation on the input data to obtain output data after the pooling operation, and the pooling operation includes: average pooling, maximum pooling or median pooling.

[0772] Furthermore, the operation unit is further configured to repeatedly train the pruned neural network until no weights can be set to zero without losing set accuracy.

[0773] A data quantification method, comprising:

[0774] Grouping the weights of a neural network;

[0775] Perform a clustering operation on each set of weights using a clustering algorithm to divide the set of weights into m classes, calculate a central weight for each class, and replace all weights in each class with the central weight, where m is a positive integer; and

[0776] The central weights are encoded to obtain a codebook and a weight dictionary.

[0777] Furthermore, the method further comprises:

[0778] Retrain the neural network. During retraining, only the codebook is trained, and the content of the weight dictionary remains unchanged.

[0779] Furthermore, the retraining adopts a back propagation algorithm.

[0780] Furthermore, the grouping includes grouping into one group, layer type grouping, inter-layer grouping and / or intra-layer grouping.

[0781] Furthermore, the clustering algorithm includes K-means, K-medoids, Clara and / or Clarans.

[0782] Furthermore, the grouping is divided into a group including:

[0783] Group all weights of a neural network together.

[0784] Furthermore, the neural network includes i convolutional layers, j fully connected layers, m long short-term memory (LSTM) layers, and a total of t different types of layers, wherein i, j, and m are integers greater than or equal to 0 and satisfy i+j+m≥1, and t is an integer greater than or equal to 1 and satisfies t=i+j+m. The grouping is a layer type grouping, including:

[0785] The weights of the neural network will be divided into t groups.

[0786] Furthermore, the grouping is an inter-layer grouping, including:

[0787] The weights of one or more convolutional layers, one or more fully connected layers, and one or more LSTM layers in the neural network are divided into one group.

[0788] Furthermore, the grouping is intra-layer grouping, including:

[0789] The convolutional layer of the neural network is regarded as a four-dimensional matrix (Nfin, Nfout, Kx, Ky), where Nfin, Nfout, Kx, Ky are positive integers, Nfin represents the number of input feature images, Nfout represents the number of output feature images, (Kx, Ky) represents the size of the convolution kernel, and the weights of the convolutional layer are divided into Nfin*Nfout*Kx*Ky / (Bfin*Bfout*Bx*By) different groups according to the group size of (Bfin, Bfout, Bx, By), where Bfin is a positive integer less than or equal to Nfin, Bfout is a positive integer less than or equal to Nfout, Bx is a positive integer less than or equal to Kx, and By is a positive integer less than or equal to Ky, or;

[0790] The fully connected layer of the neural network is regarded as a two-dimensional matrix (Nin, Nout), where Nin and Nout are positive integers, Nin represents the number of input neurons, Nout represents the number of output neurons, and there are a total of Nin*Nout weights; the fully connected layer weights are divided into (Nin*Nout) / (Bin*Bout) different groups according to the group size (Bin, Bout), where Bin is a positive integer less than or equal to Nin, and Bout is a positive integer less than or equal to Nout, or;

[0791] The LSTM layer weights of the neural network are taken as the combination of the weights of multiple fully connected layers, and the weights of the LSTM layer are composed of n fully connected layer weights, where n is a positive integer. Then each fully connected layer can be grouped according to the grouping method of the fully connected layer.

[0792] Furthermore, the grouping is divided into one group, intra-layer grouping and inter-layer grouping, including:

[0793] The convolutional layers are grouped as a group, the fully connected layers are grouped within the layers, and the LSTM layers are grouped between the layers.

[0794] Furthermore, the method for selecting the center weight of a class is to minimize the cost function J(w, w0).

[0795] Furthermore, the cost function is:

[0796]

[0797] Among them, w is the weight in a class, w0 is the central weight of the class, n is the number of weights in the class, n is a positive integer, wi is the i-th weight in the class, i is a positive integer, and 1≤i≤n.

[0798] In a twelfth aspect, an embodiment of the present invention provides a data quantization device, including:

[0799] a memory for storing operation instructions; and

[0800] The processor is used to execute the operation instructions in the memory, and when executing the operation instructions, operates according to all or part of the quantization method described in the eleventh aspect.

[0801] Furthermore, the operation instruction is a binary number, including an operation code and an address code. The operation code indicates the operation to be performed by the processor, and the address code instructs the processor to read data involved in the operation from an address in the memory.

[0802] In a thirteenth aspect, an embodiment of the present invention provides a processing device, including:

[0803] A control unit is used to receive and decode instructions and generate search control information and operation control information;

[0804] a table lookup unit, configured to receive the lookup control information, the weight dictionary, and the codebook, and perform a table lookup operation on the weight dictionary and the codebook according to the lookup control information to obtain quantized weights; and

[0805] The operation unit is used to receive the operation control information and the input neurons, and perform operation operations on the quantization weights and the input neurons according to the operation control information to obtain and output the output neurons.

[0806] Furthermore, the processing device further includes:

[0807] A preprocessing unit, configured to preprocess the input information inputted from the outside to obtain the input neurons, weight dictionary, codebook and instructions;

[0808] a storage unit for storing input neurons, weight dictionaries, codebooks, and instructions, and receiving output neurons;

[0809] a cache unit, for caching the instructions, input neurons, output neurons, weight dictionary and codebook;

[0810] A direct memory access unit is used to read and write data or instructions between the storage unit and the cache unit.

[0811] Furthermore, in the preprocessing unit, the preprocessing of the external input information includes: segmentation, Gaussian filtering, binarization, regularization and / or normalization.

[0812] Furthermore, the cache unit includes:

[0813] An instruction cache unit, configured to cache the instructions;

[0814] An input neuron cache unit, configured to cache the input neurons;

[0815] The output neuron cache unit is used to cache the output neurons.

[0816] Furthermore, the cache unit further includes:

[0817] A weight dictionary cache unit, used for caching the weight dictionary;

[0818] The code book cache unit is used to cache the code book.

[0819] Furthermore, the instructions are neural network-specific instructions.

[0820] Furthermore, the neural network-specific instructions include:

[0821] Control instructions, used to control the execution process of the neural network;

[0822] Data transfer instructions are used to complete data transfer between different storage media. The data formats include matrix, vector and scalar.

[0823] Operation instructions, used to complete the arithmetic operations of neural networks, including matrix operation instructions, vector operation instructions, scalar operation instructions, convolutional neural network operation instructions, fully connected neural network operation instructions, pooling neural network operation instructions, RBM neural network operation instructions, LRN neural network operation instructions, LCN neural network operation instructions, LSTM neural network operation instructions, RNN neural network operation instructions, RELU neural network operation instructions, PRELU neural network operation instructions, SIGMOID neural network operation instructions, TANH neural network operation instructions, and MAXOUT neural network operation instructions;

[0824] Logical instructions are used to complete the logical operations of neural networks, including vector logical operation instructions and scalar logical operation instructions.

[0825] Furthermore, the neural network-specific instructions include at least one Cambricon instruction, the Cambricon instruction including an opcode and an operand, and the Cambricon instruction includes:

[0826] Cambricon control instructions, used to control the execution process, and the Cambricon control instructions include jump instructions and conditional branch instructions;

[0827] Cambricon data transfer instructions are used to complete data transfer between different storage media, including load instructions, store instructions, and move instructions; wherein, the load instruction is used to load data from the main memory to the cache; the store instruction is used to store data from the cache to the main memory; the move instruction is used to move data between caches, caches and registers, or registers;

[0828] Cambricon operation instructions, used to perform neural network arithmetic operations, including Cambricon matrix operation instructions, Cambricon vector operation instructions, and Cambricon scalar operation instructions;

[0829] Among them, the Cambricon matrix operation instructions are used to complete matrix operations in neural networks, including matrix multiplication by vector, vector multiplication by matrix, matrix multiplication by scalar, outer product, matrix addition to matrix, and matrix subtraction from matrix; the Cambricon vector operation instructions are used to complete vector operations in neural networks, including basic vector operations, vector transcendental function operations, inner product, vector random generation, and maximum / minimum values ​​in vectors; Cambricon scalar operation instructions are used to complete scalar operations in neural networks, including basic scalar operations and scalar transcendental function operations;

[0830] Cambricon logic instructions are used for logical operations of neural networks. Cambricon logic instructions include Cambricon vector logic operation instructions and Cambricon scalar logic operation instructions;

[0831] Among them, the Cambricon vector logic operation instruction is used to complete vector comparison, vector logic operation and vector greater than merging; vector logic operations include AND, OR and NOT; the Cambricon scalar logic operation instruction is used to complete scalar comparison and scalar logic operation.

[0832] Furthermore, the Cambricon data transfer instructions support one or more of the following data organization methods: matrix, vector, and scalar;

[0833] The basic vector operations include vector addition, subtraction, multiplication and division;

[0834] Vector transcendental functions refer to functions that do not have enough coefficients to represent a polynomial equation, including exponential functions, logarithmic functions, trigonometric functions, and inverse trigonometric functions.

[0835] The basic scalar operations include scalar addition, subtraction, multiplication, and division; scalar transcendental functions refer to functions of polynomial equations that do not have enough coefficients to be polynomials, including exponential functions, logarithmic functions, trigonometric functions, and inverse trigonometric functions;

[0836] The vector comparison includes, but is not limited to, greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤), and not equal to;

[0837] The vector logic operations include AND, OR and NOT;

[0838] The scalar comparisons include, but are not limited to, greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤), and not equal to;

[0839] The scalar logic operations include AND, OR, and NOT.

[0840] Furthermore, the storage unit is also used to store unquantized weights, and the unquantized weights are directly output to the operation unit.

[0841] Furthermore, the computing unit includes:

[0842] A first operation part, configured to multiply the weight value and the input neuron, and / or;

[0843] a second operation part, comprising one or more adders, configured to add the weights and input neurons through the one or more adders, and / or;

[0844] A third operation part performs a nonlinear function operation on the weights and input neurons, wherein the nonlinear function includes an activation function, and the activation function includes sigmoid, tanh, relu and / or softmax, and / or;

[0845] The fourth operation part is used to perform a pooling operation on the weights and input neurons, wherein the pooling operation includes average pooling, maximum pooling and / or median pooling; wherein the weights include unquantized weights and / or quantized weights.

[0846] Furthermore, the second operation part includes a plurality of adders, and the plurality of adders form an addition tree to implement the step-by-step addition of weights and input neurons.

[0847] A processing method comprises the steps of:

[0848] Receive input neurons, weight dictionary, codebook and instructions;

[0849] Decoding the instruction to obtain search control information and operation control information;

[0850] According to the search control information, the weight dictionary and the code book are searched to obtain the quantized weights, and according to the operation control information, the quantized weights and the input neurons are operated to obtain the output neurons and output them.

[0851] Furthermore, before receiving the input neurons, the weight dictionary, the codebook, and the instructions, the method further comprises the steps of:

[0852] Preprocessing the input information of the external input to obtain the input neurons, weight dictionary, code book and instructions, and;

[0853] After receiving the input neurons, the weight dictionary, the codebook and the instructions, the method further comprises the steps of:

[0854] Storing input neurons, weight dictionaries, codebooks, and instructions, and storing output neurons; and caching the instructions, input neurons, and output neurons.

[0855] Furthermore, after receiving the input neurons, the weight dictionary, the codebook and the instruction, the method further comprises the step of caching the weight dictionary and the codebook.

[0856] Furthermore, the preprocessing includes segmentation, Gaussian filtering, binarization, regularization and / or normalization.

[0857] Furthermore, the instructions are neural network-specific instructions.

[0858] Furthermore, the neural network-specific instructions include:

[0859] Control instructions, used to control the execution process of the neural network;

[0860] Data transfer instructions are used to complete data transfer between different storage media. The data formats include matrix, vector and scalar.

[0861] Operation instructions, used to complete arithmetic operations of neural networks, including matrix operation instructions, vector operation instructions, scalar operation instructions, convolutional neural network operation instructions, fully connected neural network operation instructions, pooling neural network operation instructions, RBM neural network operation instructions, LRN neural network operation instructions, LCN neural network operation instructions, LSTM neural network operation instructions, RNN neural network operation instructions, RELU neural network operation instructions, PRELU neural network operation instructions, SIGMOID neural network operation instructions, TANH neural network operation instructions, and MAXOUT neural network operation instructions;

[0862] Logical instructions are used to complete the logical operations of the neural network, and the logical instructions include vector logical operation instructions and scalar logical operation instructions.

[0863] Furthermore, the neural network-specific instructions include at least one Cambricon instruction, the Cambricon instruction including an opcode and an operand, and the Cambricon instruction includes:

[0864] Cambricon control instructions, used to control the execution process, and the Cambricon control instructions include jump instructions and conditional branch instructions;

[0865] Cambricon data transmission instructions are used to complete data transmission between different storage media. The Cambricon data transmission instructions include load instructions, store instructions, and move instructions;

[0866] The load instruction is used to load data from the main memory to the cache; the store instruction is used to store data from the cache to the main memory; the move instruction is used to move data between caches or between caches and registers or between registers;

[0867] Cambricon operation instructions, used to perform neural network arithmetic operations, the Cambricon operation instructions include Cambricon matrix operation instructions, Cambricon vector operation instructions and Cambricon scalar operation instructions;

[0868] Among them, the Cambricon matrix operation instructions are used to complete matrix operations in neural networks, including matrix multiplication by vector, vector multiplication by matrix, matrix multiplication by scalar, outer product, matrix addition to matrix, and matrix subtraction from matrix; the Cambricon vector operation instructions are used to complete vector operations in neural networks, including basic vector operations, vector transcendental function operations, inner product, vector random generation, and maximum / minimum values ​​in vectors; Cambricon scalar operation instructions are used to complete scalar operations in neural networks, including basic scalar operations and scalar transcendental function operations;

[0869] Cambricon logic instructions are used for logical operations of neural networks, and the Cambricon logic instructions include Cambricon vector logic operation instructions and Cambricon scalar logic operation instructions; wherein, the Cambricon vector logic operation instructions are used to complete vector comparison, vector logic operation and vector greater than merging; vector logic operations include AND, OR and NOT; the Cambricon scalar logic operation instructions are used to complete scalar comparison and scalar logic operation.

[0870] Furthermore, the Cambricon data transmission instruction supports one or more of the following data organization methods: matrix, vector and scalar; the basic vector operations include vector addition, subtraction, multiplication and division; vector transcendental function refers to a function of a polynomial equation that is not sufficient to be a polynomial as a coefficient, including exponential function, logarithmic function, trigonometric function and inverse trigonometric function; the basic scalar operations include scalar addition, subtraction, multiplication and division; scalar transcendental function refers to a function of a polynomial equation that is not sufficient to be a polynomial as a coefficient, including exponential function, logarithmic function, trigonometric function and inverse trigonometric function; the vector comparison includes but is not limited to greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤) and not equal to; the vector logical operations include AND, OR and NOT; the scalar comparison includes but is not limited to greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤) and not equal to; the scalar logical operations include AND, OR and NOT.

[0871] Furthermore, the method further includes the steps of: receiving unquantized weights, and performing operations on the unquantized weights and input neurons according to operation control information to obtain output neurons and output them.

[0872] Furthermore, the calculation operation includes:

[0873] Adding weights and input neurons, and / or;

[0874] Multiply the weights and input neurons, and / or;

[0875] Performing a nonlinear function operation on the weights and input neurons, wherein the nonlinear function includes an activation function, and the activation function includes sigmoid, tanh, relu and / or softmax, and / or;

[0876] Performing a pooling operation on the weights and input neurons, wherein the pooling operation includes average pooling, maximum pooling and / or median pooling,

[0877] The weights include quantized weights and / or non-quantized weights.

[0878] Furthermore, the addition of weights and input neurons is implemented by one or more adders.

[0879] Furthermore, the multiple adders form an addition tree to achieve step-by-step addition of weights and input neurons.

[0880] A processing device, comprising:

[0881] A control unit is used to receive and decode instructions and generate search control information and operation control information;

[0882] a table lookup unit, configured to receive the lookup control information, the weight dictionary, and the codebook, and perform a table lookup operation on the weight dictionary and the codebook according to the lookup control information to obtain quantization weights; and

[0883] The operation unit is used to receive the operation control information, the input neurons and the quantization weights, and perform operation operations on the quantization weights and the input neurons according to the operation control information to obtain and output the output neurons.

[0884] Furthermore, the processing device further includes:

[0885] a preprocessing unit, configured to preprocess external input information to obtain the input neurons, weight dictionary, codebook, and instructions;

[0886] a storage unit, configured to store the input neurons, the weight dictionary, the codebook, and the instructions, and to receive the output neurons;

[0887] a cache unit, configured to cache the instructions, input neurons, output neurons, weight dictionaries, and codebooks; and

[0888] A direct memory access unit is used to read and write data or instructions between the storage unit and the cache unit.

[0889] Furthermore, the preprocessing unit performs preprocessing on the external input information including segmentation, Gaussian filtering, binarization, regularization and / or normalization.

[0890] Furthermore, the cache unit includes:

[0891] An instruction cache unit, configured to cache the instructions;

[0892] an input neuron buffer unit, configured to buffer the input neurons; and

[0893] The output neuron cache unit is used to cache the output neurons.

[0894] Furthermore, the cache unit further includes:

[0895] A weight dictionary cache unit, configured to cache the weight dictionary; and

[0896] The code book cache unit is used to cache the code book.

[0897] Furthermore, the instructions are neural network-specific instructions.

[0898] Furthermore, the neural network-specific instructions include:

[0899] Control instructions, used to control the execution process of the neural network;

[0900] Data transfer instructions are used to complete data transfer between different storage media. The data formats include matrix, vector and scalar.

[0901] Operation instructions for performing neural network arithmetic operations, including matrix operation instructions, vector operation instructions, scalar operation instructions, convolutional neural network operation instructions, fully connected neural network operation instructions, pooling neural network operation instructions, restricted Boltzmann machine (RBM) neural network operation instructions, nearest neighbor normalization (LRN) neural network operation instructions, local contrast normalization (LCN) neural network operation instructions, long short-term memory (LSTM) neural network operation instructions, recurrent neural network (RNN) operation instructions, linear rectification unit (RELU) neural network operation instructions, parameterized linear rectification unit (PRELU) neural network operation instructions, S-shaped growth curve (SIGMOID) neural network operation instructions, hyperbolic tangent function (TANH) neural network operation instructions, and maximum output (MAXOUT) neural network operation instructions; and

[0902] Logical instructions are used to complete the logical operations of neural networks, including vector logical operation instructions and scalar logical operation instructions.

[0903] Furthermore, the neural network-specific instructions include at least one Cambricon instruction, the Cambricon instruction including an opcode and an operand, and the Cambricon instruction includes:

[0904] Cambricon control instructions, used to control the execution process, and the Cambricon control instructions include jump instructions and conditional branch instructions;

[0905] Cambricon data transfer instructions are used to complete data transfer between different storage media, including load instructions, store instructions, and move instructions; wherein, the load instruction is used to load data from the main memory to the cache; the store instruction is used to store data from the cache to the main memory; the move instruction is used to move data between caches, between caches and registers, or between registers;

[0906] Cambricon operation instructions are used to perform neural network arithmetic operations, including Cambricon matrix operation instructions, Cambricon vector operation instructions, and Cambricon scalar operation instructions; wherein, the Cambricon matrix operation instructions are used to perform matrix operations in the neural network, including matrix multiplication by vector, vector multiplication by matrix, matrix multiplication by scalar, outer product, matrix addition to matrix, and matrix subtraction from matrix; the Cambricon vector operation instructions are used to perform vector operations in the neural network, including basic vector operations, vector transcendental function operations, inner product, vector random generation, and maximum / minimum value in vector; Cambricon scalar operation instructions are used to perform scalar operations in the neural network, including basic scalar operations and scalar transcendental function operations; and

[0907] Cambricon logic instructions are used for logical operations of neural networks, and the logical operations include Cambricon vector logic operation instructions and Cambricon scalar logic operation instructions; wherein, the Cambricon vector logic operation instructions include vector comparison, vector logic operation and vector greater than merging; vector logic operations include AND, OR and NOT; the Cambricon scalar logic operations include scalar comparison and scalar logic operation.

[0908] Furthermore, the Cambricon data transmission instruction supports one or more of the following data organization methods: matrix, vector and scalar; the basic vector operations include vector addition, subtraction, multiplication and division; vector transcendental function refers to a function of a polynomial equation that is not sufficient to have a polynomial as a coefficient, including exponential function, logarithmic function, trigonometric function and inverse trigonometric function; the basic scalar operations include scalar addition, subtraction, multiplication and division; scalar transcendental function refers to a function of a polynomial equation that is not sufficient to have a polynomial as a coefficient, including exponential function, logarithmic function, trigonometric function and inverse trigonometric function; the vector comparison includes greater than, less than, equal to, greater than or equal to, less than or equal to and not equal to; the vector logical operations include AND, OR and NOT; the scalar comparison includes greater than, less than, equal to, greater than or equal to, less than or equal to and not equal to; the scalar logical operations include AND, OR and NOT.

[0909] Furthermore, the storage unit is also used to store unquantized weights, and the unquantized weights are directly output to the operation unit.

[0910] Furthermore, the computing unit includes:

[0911] A first operation part is used to multiply the weight value and the input neuron; and / or,

[0912] A second operation part includes one or more adders, configured to add the weights and input neurons through the one or more adders; and / or,

[0913] The third operation part performs a nonlinear function operation on the weights and input neurons, wherein the nonlinear function includes an activation function, and the activation function includes sigmoid, tanh, relu and / or softmax; and / or,

[0914] a fourth operation part, configured to perform a pooling operation on the weights and input neurons, wherein the pooling operation includes average pooling, maximum pooling, and / or median pooling;

[0915] The weights include unquantized weights and / or quantized weights.

[0916] Furthermore, the second operation part includes a plurality of adders, and the plurality of adders form an addition tree to implement the step-by-step addition of weights and input neurons.

[0917] A processing method comprises the steps of:

[0918] Receive input neurons, weight dictionary, codebook and instructions;

[0919] Decoding the instruction to obtain search control information and operation control information; and

[0920] According to the search control information, the weight dictionary and the code book are searched to obtain the quantized weights, and according to the operation control information, the quantized weights and the input neurons are operated to obtain and output the output neurons.

[0921] Furthermore, before receiving the input neurons, weight dictionary, codebook and instructions, the method further includes the steps of:

[0922] Preprocessing the external input information to obtain the input neurons, weight dictionary, code book and instructions; and

[0923] After receiving the input neurons, the weight dictionary, the code book and the instruction, the method further comprises the steps of:

[0924] Storing the input neurons, weight dictionary, codebook and instructions, and storing the output neurons; and caching the instructions, input neurons and output neurons.

[0925] Furthermore, after receiving the input neurons, the weight dictionary, the codebook, and the instructions, the method further comprises the steps of:

[0926] Cache weight dictionary and codebook.

[0927] Furthermore, the preprocessing includes segmentation, Gaussian filtering, binarization, regularization and / or normalization.

[0928] Furthermore, the instructions are neural network-specific instructions.

[0929] Furthermore, the neural network-specific instructions include:

[0930] Control instructions, used to control the execution process of the neural network;

[0931] Data transfer instructions are used to complete data transfer between different storage media. The data formats include matrix, vector and scalar.

[0932] Operation instructions for performing neural network arithmetic operations, including matrix operation instructions, vector operation instructions, scalar operation instructions, convolutional neural network operation instructions, fully connected neural network operation instructions, pooling neural network operation instructions, restricted Boltzmann machine (RBM) neural network operation instructions, nearest neighbor normalization (LRN) neural network operation instructions, LCN local contrast normalization (LCN) neural network operation instructions, long short-term memory (LSTM) neural network operation instructions, recurrent neural network (RNN) operation instructions, linear rectifier unit (RELU) neural network operation instructions, parameterized linear rectifier unit (PRELU) neural network operation instructions, S-shaped growth curve (SIGMOID) neural network operation instructions, hyperbolic tangent function (TANH) neural network operation instructions, and maximum output (MAXOUT) neural network operation instructions; and

[0933] Logical instructions are used to complete the logical operations of neural networks, including vector logical operation instructions and scalar logical operation instructions.

[0934] Furthermore, the neural network-specific instructions include at least one Cambricon instruction, the Cambricon instruction including an opcode and an operand, and the Cambricon instruction includes:

[0935] Cambricon control instructions are used to control the execution process, and the Cambricon control instructions include jump instructions and conditional branch instructions;

[0936] Cambricon data transfer instructions are used to complete data transfer between different storage media, including load instructions, store instructions, and move instructions. Among them, the load instruction is used to load data from the main memory to the cache; the store instruction is used to store data from the cache to the main memory; the move instruction is used to move data between caches, caches and registers, or registers.

[0937] Cambricon operation instructions are used to perform neural network arithmetic operations, including Cambricon matrix operation instructions, Cambricon vector operation instructions, and Cambricon scalar operation instructions; wherein, the Cambricon matrix operation instructions are used to perform matrix operations in the neural network, including matrix multiplication by vector, vector multiplication by matrix, matrix multiplication by scalar, outer product, matrix addition to matrix, and matrix subtraction from matrix; the Cambricon vector operation instructions are used to perform vector operations in the neural network, including basic vector operations, vector transcendental function operations, inner product, vector random generation, and maximum / minimum value in vector; Cambricon scalar operation instructions are used to perform scalar operations in the neural network, including basic scalar operations and scalar transcendental function operations; and

[0938] Cambricon logic instructions are used for logical operations of neural networks, and the logical operations include Cambricon vector logic operation instructions and Cambricon scalar logic operation instructions; wherein, the Cambricon vector logic operation instructions include vector comparison, vector logic operation and vector greater than merging; vector logic operations include AND, OR and NOT; the Cambricon scalar logic operations include scalar comparison and scalar logic operation.

[0939] Furthermore, the Cambricon data transmission instruction supports one or more of the following data organization methods: matrix, vector and scalar; the basic vector operations include vector addition, subtraction, multiplication and division; vector transcendental function refers to a function of a polynomial equation that is not sufficient to have a polynomial as a coefficient, including exponential function, logarithmic function, trigonometric function and inverse trigonometric function; the basic scalar operations include scalar addition, subtraction, multiplication and division; scalar transcendental function refers to a function of a polynomial equation that is not sufficient to have a polynomial as a coefficient, including exponential function, logarithmic function, trigonometric function and inverse trigonometric function; the vector comparison includes greater than, less than, equal to, greater than or equal to, less than or equal to and not equal to; the vector logical operations include AND, OR and NOT; the scalar comparison includes greater than, less than, equal to, greater than or equal to, less than or equal to and not equal to; the scalar logical operations include AND, OR and NOT.

[0940] Furthermore, the method further comprises the steps of:

[0941] An unquantized weight value is received, and according to the operation control information, an operation is performed on the unquantized weight value and the input neuron to obtain and output the output neuron.

[0942] Furthermore, the calculation operation includes:

[0943] Adding the weights and input neurons; and / or

[0944] Multiply the weights and input neurons; and / or

[0945] Performing a nonlinear function operation on the weights and input neurons, wherein the nonlinear function includes an activation function, and the activation function includes sigmoid, tanh, relu and / or softmax; and / or

[0946] Performing a pooling operation on the weights and input neurons, wherein the pooling operation includes average pooling, maximum pooling and / or median pooling,

[0947] The weights include quantized weights and / or non-quantized weights.

[0948] Furthermore, the addition of weights and input neurons is implemented by one or more adders.

[0949] Furthermore, the multiple adders form an addition tree to achieve step-by-step addition of weights and input neurons.

[0950] A data quantification method comprises the steps of:

[0951] Grouping the weights of a neural network;

[0952] Perform a clustering operation on each set of weights using a clustering algorithm, dividing the set of weights into m classes, calculating the central weight of each class, and replacing all weights in each class with the central weight corresponding to the class, where m is a positive integer; and

[0953] The central weights are encoded to obtain a codebook and a weight dictionary.

[0954] Furthermore, the method further comprises the steps of:

[0955] Retrain the neural network. During retraining, only the codebook is trained, and the content of the weight dictionary remains unchanged.

[0956] Furthermore, the retraining adopts a back propagation algorithm.

[0957] Furthermore, the grouping includes grouping into one group, layer type grouping, inter-layer grouping and / or intra-layer grouping.

[0958] Furthermore, the clustering algorithm includes K-means, K-medoids, Clara and / or Clarans.

[0959] Furthermore, the grouping is grouping into a group, specifically including:

[0960] Group all weights of a neural network together.

[0961] Furthermore, the neural network includes i convolutional layers, j fully connected layers, m long short-term memory (LSTM) layers, one convolutional layer, j fully connected layers, and a total of t different types of layers, where i, j, and m are all integers greater than 0 and satisfy i+j+m≥1, and t is a positive integer greater than or equal to 1 and satisfies t=i+j+m. The grouping is a layer type grouping, including:

[0962] The weights of the neural network will be divided into t groups.

[0963] Furthermore, the grouping is an inter-layer grouping, including:

[0964] The weights of one or more convolutional layers, one or more fully connected layers, and one or more LSTM layers in the neural network are divided into one group.

[0965] Furthermore, the grouping is intra-layer grouping, including:

[0966] The convolutional layer of the neural network is regarded as a four-dimensional matrix (Nfin, Nfout, Kx, Ky), where Nfin, Nfout, Kx, Ky are positive integers, Nfin represents the number of input feature images, Nfout represents the number of output feature images, (Kx, Ky) represents the size of the convolution kernel, and the weights of the convolutional layer are divided into Nfin*Nfout*Kx*Ky / (Bfin*Bfout*Bx*By) different groups according to the group size of (Bfin, Bfout, Bx, By), where Bfin is a positive integer less than or equal to Nfin, Bfout is a positive integer less than or equal to Nfout, Bx is a positive integer less than or equal to Kx, and By is a positive integer less than or equal to Ky, or;

[0967] The fully connected layer of the neural network is regarded as a two-dimensional matrix (Nin, Nout), where Nin and Nout are positive integers, Nin represents the number of input neurons, Nout represents the number of output neurons, and there are a total of Nin*Nout weights; the fully connected layer weights are divided into (Nin*Nout) / (Bin*Bout) different groups according to the group size (Bin, Bout), where Bin is a positive integer less than or equal to Nin, and Bout is a positive integer less than or equal to Nout, or;

[0968] The LSTM layer weights of the neural network are taken as the combination of the weights of multiple fully connected layers, and the weights of the LSTM layer are composed of n fully connected layer weights, where n is a positive integer. Then each fully connected layer can be grouped according to the grouping method of the fully connected layer.

[0969] Furthermore, the grouping is divided into one group, intra-layer grouping and inter-layer grouping, including:

[0970] The convolutional layers are grouped as a group, the fully connected layers are grouped within the layers, and the LSTM layers are grouped between the layers.

[0971] Furthermore, the method for selecting the center weight of a class is to minimize the cost function J(w, w0).

[0972] Furthermore, the cost function is:

[0973]

[0974] Among them, w is the weight in a class, w0 is the central weight of the class, n is the number of weights in the class, n is a positive integer, wi is the i-th weight in the class, i is a positive integer, and 1≤i≤n.

[0975] A data quantization device, comprising:

[0976] A memory, for storing operation instructions;

[0977] The processor is used to execute the operation instructions in the memory, and operates according to the above-mentioned quantization method when executing the operation instructions.

[0978] Furthermore, the operation instruction is a binary number, including an operation code and an address code. The operation code indicates the operation to be performed by the processor, and the address code instructs the processor to read data involved in the operation from an address in the memory.

[0979] A processing device, comprising:

[0980] A control unit, configured to receive and decode instructions to obtain search control information and operation control information;

[0981] a table lookup unit, configured to receive the lookup control information, the weight dictionary, and the codebook, and perform a table lookup operation on the weight dictionary and the codebook according to the lookup control information to obtain quantization weights; and

[0982] The operation unit is used to receive the operation control information, the quantization weight and the input neuron, and perform an operation on the quantization weight and the input neuron according to the operation control information to obtain and output the output neuron.

[0983] Furthermore, the processing device further includes:

[0984] A preprocessing unit, configured to preprocess the input information inputted from the outside to obtain the input neurons, weight dictionary, codebook and instructions;

[0985] a storage unit for storing the input neurons, weight dictionary, codebook and instructions, and receiving output neurons;

[0986] a cache unit, configured to cache the instructions, input neurons, output neurons, weight dictionaries, and codebooks; and

[0987] A direct memory access unit is used to read and write data or instructions between the storage unit and the cache unit.

[0988] Furthermore, the preprocessing operations performed by the preprocessing unit on the external input information include: segmentation, Gaussian filtering, binarization, regularization and / or normalization.

[0989] Furthermore, the cache unit includes:

[0990] An instruction cache unit, configured to cache the instructions;

[0991] an input neuron buffer unit, configured to buffer the input neurons; and

[0992] The output neuron cache unit is used to cache the output neurons.

[0993] Furthermore, the cache unit further includes: a weight dictionary cache for caching the weight dictionary; and a codebook cache for caching the codebook.

[0994] Furthermore, the instructions are neural network-specific instructions.

[0995] Furthermore, the neural network-specific instructions include:

[0996] Control instructions, used to control the execution process of the neural network;

[0997] Data transfer instructions are used to complete data transfer between different storage media. The data formats include matrix, vector and scalar.

[0998] Operation instructions, used to complete the arithmetic operations of neural networks, including matrix operation instructions, vector operation instructions, scalar operation instructions, convolutional neural network operation instructions, fully connected neural network operation instructions, pooling neural network operation instructions, RBM neural network operation instructions, LRN neural network operation instructions, LCN neural network operation instructions, LSTM neural network operation instructions, RNN neural network operation instructions, RELU neural network operation instructions, PRELU neural network operation instructions, SIGMOID neural network operation instructions, TANH neural network operation instructions, and MAXOUT neural network operation instructions;

[0999] And logic instructions, used to complete the logical operations of neural networks, including vector logic operation instructions and scalar logic operation instructions.

[1000] Furthermore, the neural network-specific instructions include at least one Cambricon instruction, which includes an opcode and an operand, and the Cambricon instructions include: Cambricon control instructions for controlling the execution process, and the Cambricon control instructions include jump instructions and conditional branch instructions; Cambricon data transmission instructions for completing data transmission between different storage media, including load instructions, store instructions, and move instructions; wherein, the load instruction is used to load data from the main memory to the cache; the store instruction is used to store data from the cache to the main memory; the move instruction is used to move data between the cache and the cache or between the cache and the register or between the register; Cambricon operation instructions are used to complete neural network arithmetic operations, including Cambricon matrix operation instructions, Cambricon vector operation instructions and Cambricon scalar operation instructions; wherein, the Cambricon n matrix operation instructions are used to complete matrix operations in neural networks, including matrix multiplication by vector, vector multiplication by matrix, matrix multiplication by scalar, outer product, matrix addition to matrix and matrix subtraction from matrix; the Cambricon vector operation instructions are used to complete vector operations in neural networks, including basic vector operations, vector transcendental function operations, inner product, vector random generation and maximum / minimum values ​​in vectors; Cambricon scalar operation instructions are used to complete scalar operations in neural networks, including basic scalar operations and scalar transcendental function operations; Cambricon logic instructions are used for logical operations of neural networks, and logic operation instructions include Cambricon vector logic operation instructions and Cambricon scalar logic operation instructions; wherein, the Cambricon vector logic operation instructions are used to complete vector comparison, vector logic operation and vector greater than merging; vector logic operations include AND, OR and NOT; the Cambricon scalar logic operation instructions are used to complete scalar comparison and scalar logic operations.

[1001] Furthermore, the Cambricon data transmission instruction supports one or more of the following data organization methods: matrix, vector and scalar; the basic vector operations include vector addition, subtraction, multiplication and division; vector transcendental function refers to a function of a polynomial equation that is not sufficient to be a polynomial as a coefficient, including exponential function, logarithmic function, trigonometric function and inverse trigonometric function; the basic scalar operations include scalar addition, subtraction, multiplication and division; scalar transcendental function refers to a function of a polynomial equation that is not sufficient to be a polynomial as a coefficient, including exponential function, logarithmic function, trigonometric function and inverse trigonometric function; the vector comparison includes greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤) and not equal to; the vector logical operations include AND, OR and NOT; the scalar comparison includes greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤) and not equal to; the scalar logical operations include AND, OR and NOT.

[1002] Furthermore, the storage unit is also used to store unquantized weights, and the unquantized weights are directly output to the operation unit.

[1003] Furthermore, the operation unit includes: a first operation part, used to multiply the weight and input neuron; and / or a second operation part, including one or more adders, used to add the weight and input neuron through one or more adders; and / or a third operation part, used to perform nonlinear function operation on the weight and input neuron, the nonlinear function includes an activation function, the activation function includes sigmoid, tanh, relu and / or softmax; and / or a fourth operation part, used to perform pooling operation on the weight and input neuron, the pooling operation includes average pooling, maximum pooling and / or median pooling; wherein, the weight is an unquantized weight and / or a quantized weight.

[1004] Furthermore, the second operation part includes a plurality of adders, and the plurality of adders form an addition tree to implement the step-by-step addition of weights and input neurons.

[1005] A processing method comprises the steps of:

[1006] Receive input neurons, weight dictionary, codebook and instructions;

[1007] Decoding the instruction to obtain search control information and operation control information; and

[1008] According to the search control information, the weight dictionary and the code book are searched to obtain the quantized weights, and according to the operation control information, the quantized weights and the input neurons are operated to obtain and output the output neurons.

[1009] Furthermore, before receiving the input neurons, the weight dictionary, the codebook, and the instructions, the method further comprises the steps of:

[1010] Preprocessing the input information of the external input to obtain the input neurons, weight dictionary, code book and instructions;

[1011] After receiving the input neurons, the weight dictionary, the codebook, and the instructions, the method further comprises the steps of:

[1012] Storing the input neurons, weight dictionary, codebook and instructions, and storing the output neurons; and caching the instructions, input neurons and output neurons.

[1013] Furthermore, after receiving the input neurons, the weight dictionary, the codebook and the instruction, the method further comprises the step of caching the weight dictionary and the codebook.

[1014] Furthermore, the preprocessing includes segmentation, Gaussian filtering, binarization, regularization and / or normalization.

[1015] Furthermore, the instructions are neural network-specific instructions.

[1016] Furthermore, the neural network-specific instructions include: control instructions for controlling the execution process of the neural network; data transmission instructions for completing data transmission between different storage media, and the data formats include matrices, vectors and scalars; operation instructions for completing arithmetic operations of the neural network, including matrix operation instructions, vector operation instructions, scalar operation instructions, convolutional neural network operation instructions, fully connected neural network operation instructions, pooling neural network operation instructions, RBM neural network operation instructions, LRN neural network operation instructions, LCN neural network operation instructions, LSTM neural network operation instructions, RNN neural network operation instructions, RELU neural network operation instructions, PRELU neural network operation instructions, SIGMOID neural network operation instructions, TANH neural network operation instructions, MAXOUT neural network operation instructions; and logic instructions for completing logical operations of the neural network, including vector logic operation instructions and scalar logic operation instructions.

[1017] Furthermore, the neural network-specific instructions include at least one Cambricon instruction, which includes an opcode and an operand, and the Cambricon instructions include: Cambricon control instructions for controlling the execution process, and the Cambri con control instructions include jump instructions and conditional branch instructions; Cambricon data transmission instructions for completing data transmission between different storage media, including load instructions, store instructions, and move instructions; wherein, the load instructions are used to load data from the main memory to the cache; the store instructions are used to store data from the cache to the main memory; the move instructions are used to move data between the cache and the cache or the cache and the register or the register and the register; Cambricon operation instructions are used to complete neural network arithmetic operations, including Cambricon matrix operation instructions, Cambricon vector operation instructions and Cambricon scalar operation instructions; wherein, the Cambricon matrix operation instructions are used to complete matrix operations in the neural network, including matrix multiplication by vector, vector multiplication by matrix, matrix multiplication by scalar, outer product, matrix addition by matrix and Matrix minus matrix; the Cambricon vector operation instructions are used to complete vector operations in neural networks, including basic vector operations, vector transcendental function operations, inner products, vector random generation and maximum / minimum values ​​in vectors; Cambricon scalar operation instructions are used to complete scalar operations in neural networks, including basic scalar operations and scalar transcendental function operations; and Cambricon logic instructions are used for logic operations in neural networks, and logic operation instructions include Cambricon vector logic operation instructions and Cambricon scalar logic operation instructions; wherein, the Cambricon vector logic operation instructions are used to complete vector comparison, vector logic operation and vector greater than merging; vector logic operations include AND, OR and NOT; the Cambricon scalar logic operation instructions are used to complete scalar comparison and scalar logic operations.

[1018] Furthermore, the Cambricon data transmission instruction supports one or more of the following data organization methods: matrix, vector and scalar; the basic vector operations include vector addition, subtraction, multiplication and division; vector transcendental function refers to a function of a polynomial equation that is not sufficient to be a polynomial as a coefficient, including exponential function, logarithmic function, trigonometric function and inverse trigonometric function; the basic scalar operations include scalar addition, subtraction, multiplication and division; scalar transcendental function refers to a function of a polynomial equation that is not sufficient to be a polynomial as a coefficient, including exponential function, logarithmic function, trigonometric function and inverse trigonometric function; the vector comparison includes greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤) and not equal to; the vector logical operations include AND, OR and NOT; the scalar comparison includes greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤) and not equal to; the scalar logical operations include AND, OR and NOT.

[1019] Furthermore, the method further comprises the steps of:

[1020] The unquantized weight value is received, and according to the operation control information, the unquantized weight value and the input neuron are operated to obtain and output the output neuron.

[1021] Furthermore, the calculation operation includes: adding the weights and input neurons; and / or multiplying the weights and input neurons; and / or performing a nonlinear function operation on the weights and input neurons, the nonlinear function includes an activation function, the activation function includes sigmoid, tanh, relu and / or softmax; and / or performing a pooling operation on the weights and input neurons, the pooling operation includes average pooling, maximum pooling and / or median pooling, wherein the weights include quantized weights and / or non-quantized weights.

[1022] Furthermore, the addition of weights and input neurons is implemented by one or more adders.

[1023] Furthermore, the multiple adders form an addition tree to achieve step-by-step addition of weights and input neurons.

[1024] A data compression method, comprising:

[1025] Coarse-grained pruning of the weights of the neural network includes: selecting M weights from the neural network according to a sliding window, and when the M weights meet a preset condition, setting all or part of the M weights to zero, where M is an integer greater than 0; performing a first retraining on the neural network, and maintaining the weights that have been set to zero during the training process at zero;

[1026] The weights of the neural network are quantified, including: grouping the weights of the neural network, clustering each group of weights according to a clustering algorithm, calculating the central weight of each class, and replacing all weights in each class with the central weight.

[1027] Furthermore, after quantizing the weights of the neural network, the method further includes:

[1028] The central weight is encoded to obtain a codebook and a weight dictionary.

[1029] Furthermore, after encoding the center weight, the method further includes:

[1030] The neural network is trained a second time.

[1031] Furthermore, when the neural network is trained for the second time, only the codebook is trained, and the content of the weight dictionary remains unchanged.

[1032] Furthermore, the preset conditions are:

[1033] The amount of information of the M weights is less than a first preset threshold.

[1034] Furthermore, the information amount of the M weights is the arithmetic mean of the absolute values ​​of the M weights, the geometric mean of the absolute values ​​of the M weights, or the maximum value of the M weights, the first preset threshold is the first threshold, the second threshold, or the third threshold, and the information amount of the M weights is less than the first preset threshold, including:

[1035] The arithmetic mean of the absolute values ​​of the M weights is less than the first threshold, or the geometric mean of the absolute values ​​of the M weights is less than the second threshold, or the maximum value of the M weights is less than the third threshold.

[1036] Furthermore, the method further comprises:

[1037] Repeatedly using a sliding window to select M weights from a neural network, and when the M weights meet a preset condition, setting all or part of the M weights to zero; and performing a first retraining on the neural network until no weight can be set to zero without losing the set accuracy.

[1038] Furthermore, the setting accuracy is x%, where x is between 0 and 5.

[1039] Furthermore, the neural network includes a fully connected layer, a convolutional layer and / or a long short-term memory (LSTM) layer, wherein the weight of the fully connected layer is a two-dimensional matrix (Nin, Nout), where Nin is the number of input neurons, Nout is the number of output neurons, and the fully connected layer has Nin*Nout weights; the weight of the convolutional layer is a four-dimensional matrix (Nfin, Nfout, Kx, Ky), where Nfin is the number of input feature images, Nfout is the number of output feature images, (Kx, Ky) is the size of the convolution kernel. The convolution layer has Nfin*Nfout*Kx*Ky weights. The weight of the LSTM layer is composed of the weights of m fully connected layers, where m is an integer greater than 0. The weight of the i-th fully connected layer is (Nin_i, Nout_i), where i is an integer greater than 0 and less than or equal to m. Nin_i represents the number of input neurons of the i-th fully connected layer weight, and Nout_i represents the number of output neurons of the i-th fully connected layer weight. The coarse-grained pruning unit is specifically used to:

[1040] When a coarse-grained pruning operation is performed on the weights of the fully connected layer, the size of the sliding window is a sliding window of Bin*Bout, where Bin is an integer greater than 0 and less than or equal to Nin, and Bout is an integer greater than 0 and less than or equal to Nout;

[1041] The sliding window can slide along the direction of Bin according to the step size Sin, or along

[1042] Slide in the direction of Bout according to the step size Sout, where Sin is a positive integer greater than 0 and less than or equal to Bin, and Sout is a positive integer greater than 0 and less than or equal to Bout;

[1043] Selecting M values ​​from the Nin*Nout weights through the sliding window, and when the M weights meet the preset condition, setting all or part of the M weights to zero, wherein M=Bin*Bout;

[1044] When the coarse-grained pruning operation is performed on the weights of the convolutional layer, the sliding window is a four-dimensional sliding window of size Bfin*Bfout*Bx*By, where Bfin is an integer greater than 0 and less than or equal to Nfin, Bfout is an integer greater than 0 and less than or equal to Nfout, Bx is an integer greater than 0 and less than or equal to Kx, and By is an integer greater than 0 and less than or equal to Ky;

[1045] Slide the sliding window along the direction of Bfin with a step size Sfin, or slide along the direction of Bfout with a step size Sfout, or slide along the direction of Bx with a step size S, or slide along the direction of By with a step size Sy, where Sfin is an integer greater than 0 and less than or equal to Bfin, Sfout is an integer greater than 0 and less than or equal to Bfout, Sx is an integer greater than 0 and less than or equal to Bx, and Sy is an integer greater than 0 and less than or equal to By;

[1046] Selecting M weights from the Nfin*Nfout*Kx*Ky weights through the sliding window, and setting all or part of the M weights to zero when the M weights meet the preset condition, wherein M=Bfin*Bfout*Bx*By;

[1047] When the coarse-grained pruning operation is performed on the weights of the LSTM layer, the size of the sliding window is Bin_i*Bout_i, where Bin_i is an integer greater than 0 and less than or equal to Nin_i, and Bout_i is an integer greater than 0 and less than or equal to Nout_i;

[1048] Slide the sliding window along the direction of Bin_i according to the step size Sin_i, or slide along the direction of Bout_i according to the step size Sout_i, where Sin_i is a positive integer greater than 0 and less than or equal to Bin_i, and Sout_i is a positive integer greater than 0 and less than or equal to Bout_i;

[1049] M weights are selected from the Bin_i*Bout_i weights through the sliding window, and when the M weights meet the preset condition, all or part of the M weights are set to zero, wherein M=Bin_i*Bout_i.

[1050] Furthermore, the first training adopts a back propagation algorithm, and the weights that have been set to zero during the training process remain zero.

[1051] Furthermore, the weights of the neural network are grouped in the following ways:

[1052] grouping the weights of the neural network into a group, and / or;

[1053] grouping the weights of the neural network by layer type, and / or;

[1054] Grouping of neural network weights between and / or within layers.

[1055] Furthermore, grouping the weights of the neural network according to layer type includes:

[1056] Divide the weights of all convolutional layers, all fully connected layers, and all LSTM layers in the neural network into one group.

[1057] Furthermore, the grouping of the weights of the neural network according to the layers includes:

[1058] The weights of one or more convolutional layers, the weights of one or more fully connected layers, and the weights of one or more long short-term memory network layers in the neural network are divided into one group.

[1059] Furthermore, grouping the weights of the neural network according to the layer includes:

[1060] The weights within a layer of the neural network are divided, and each part after division is divided into a group.

[1061] Furthermore, the clustering algorithm includes K-means, K-medoids, Clara and / or Clarans.

[1062] Furthermore, the center weight selection method is to minimize the cost function J(w, w0).

[1063] Furthermore, the cost function satisfies the following conditions:

[1064]

[1065] Where w is the total weight of the class, w0 is the center weight, n is the number of weights in the class, wi is the i-th weight in the class, and i is an integer greater than 0 and less than or equal to n.

[1066] Furthermore, the second training of the clustered and encoded neural network includes:

[1067] The clustered and encoded neural network is retrained using the back-propagation algorithm. The weights that have been set to 0 during the training process will remain 0, and only the weight codebook is trained, not the weight dictionary.

[1068] A data compression device, comprising:

[1069] A memory, for storing operation instructions;

[1070] The processor is used to execute the operation instructions in the memory, and when executing the operation instructions, operates in accordance with all or part of the data compression method described in aspect 22.

[1071] A data compression method, comprising:

[1072] Coarse-grained pruning of the weights of the neural network includes: selecting M weights from the neural network according to a sliding window, and when the M weights meet a preset condition, setting all or part of the M weights to zero, where M is an integer greater than 0; performing a first retraining on the neural network, and maintaining the weights that have been set to zero during the training process at zero;

[1073] The weights of the neural network are quantified, including: grouping the weights of the neural network, clustering each group of weights according to a clustering algorithm, calculating the central weight of each class, and replacing all weights in each class with the central weight corresponding to the class.

[1074] Furthermore, after quantizing the weights of the neural network, the method further includes:

[1075] The central weight is encoded to obtain a codebook and a weight dictionary.

[1076] Furthermore, after encoding the center weight, the method further includes:

[1077] Perform the second training of the neural network.

[1078] Furthermore, when the neural network is trained for the second time, only the codebook is trained, and the content of the weight dictionary remains unchanged.

[1079] Furthermore, the preset conditions are:

[1080] The amount of information of the M weights is less than a first preset threshold.

[1081] Furthermore, the information amount of the M weights is the arithmetic mean of the absolute values ​​of the M weights, the geometric mean of the absolute values ​​of the M weights, or the maximum value of the M weights, the first preset threshold is the first threshold, the second threshold, or the third threshold, and the information amount of the M weights is less than the first preset threshold, including:

[1082] The arithmetic mean of the absolute values ​​of the M weights is less than the first threshold, or the geometric mean of the absolute values ​​of the M weights is less than the second threshold, or the maximum value of the M weights is less than the third threshold.

[1083] Furthermore, the method also includes: repeatedly using a sliding window to select M weights from a neural network, and when the M weights meet a preset condition, setting all or part of the M weights to zero; and performing a first training on the neural network until no weight can be set to zero without losing the set accuracy.

[1084] Furthermore, the setting accuracy is x%, where x is between 0 and 5.

[1085] Furthermore, the coarse-grained pruning of the weights of the neural network includes:

[1086] The weights of the fully connected layer of the neural network are pruned, or the weights of the convolutional layer of the neural network are pruned, or the weights of the long short-term memory (LSTM) layer of the neural network are pruned.

[1087] Furthermore, the weights of the fully connected layer of the neural network are a two-dimensional matrix (Nin, Nout), where Nin is the number of input neurons, Nout is the number of output neurons, the fully connected layer has Nin*Nout weights, and the size of the sliding window is Bin*Bout, where Bin is an integer greater than 0 and less than or equal to Nin, and Bout is an integer greater than 0 and less than or equal to Nout; pruning the weights of the fully connected layer of the neural network includes:

[1088] The sliding window can slide along the direction of Bin with a step size Sin, or can slide along the direction of Bout with a step size Sout, where Sin is an integer greater than 0 and less than or equal to Bin, and Sout is an integer greater than 0 and less than or equal to Bout;

[1089] M weights are selected from the Nin*Nout weights through the sliding window; when the M weights meet the preset condition, all or part of the M weights are set to zero, wherein M=Bin*Bout.

[1090] Furthermore, the weights of the convolutional layer of the neural network are a four-dimensional matrix (Nfin, Nfout, Kx, Ky), where Nfin is the number of input feature images, Nfout is the number of output feature images, (Kx, Ky) is the size of the convolution kernel, the convolutional layer has Nfin*Nfout*Kx*Ky weights, and the sliding window is a four-dimensional sliding window of size Bfin*Bfout*Bx*By, where Bfin is an integer greater than 0 and less than or equal to Nfin, Bfout is an integer greater than 0 and 1 less than or equal to Nfout, Bx is an integer greater than 0 and less than or equal to Kx, and By is an integer greater than 0 and less than or equal to Ky; pruning the weights of the convolutional layer of the neural network includes:

[1091] Slide the sliding window along the direction of Bfin with a step size Sfin, or slide along the direction of Bfout with a step size Sfout, or slide along the direction of Bx with a step size Sx, or slide along the direction of By with a step size Sy, where Sfin is an integer greater than 0 and less than or equal to Bfin, Sfout is an integer greater than 0 and less than or equal to Bfout, Sx is an integer greater than 0 and less than or equal to Bx, and Sy is an integer greater than 0 and less than or equal to By;

[1092] M weights are selected from the Nfin*Nfout*Kx*Ky weights through the sliding window; when the M weights meet the preset conditions, all or part of the M weights are set to zero, wherein M = Bfin*Bfout*Bx*By.

[1093] Furthermore, the weights of the LSTM layer of the neural network are composed of m fully connected layer weights, where m is a positive integer greater than 0, and the weight of the i-th fully connected layer is a two-dimensional matrix (Nin_i, Nout_i), where i is an integer greater than 0 and less than or equal to m, Nin_i represents the number of input neurons of the i-th fully connected layer weight, and Nout_i represents the number of output neurons of the i-th fully connected layer weight; the size of the sliding window is Bin_i*Bout_i, where Bin_i is an integer greater than 0 and less than or equal to Nin_i, and Bout_i is an integer greater than 0 and less than or equal to Nout_i; pruning the LSTM layer of the neural network includes:

[1094] Slide the sliding window along the direction of Bin_i with a step size Sin_i, or slide along the direction of Bout_i with a step size Sout_i, where Sin_i is an integer greater than 0 and less than or equal to Bin_i, and Sout_i is an integer greater than 0 and less than or equal to Bout_i;

[1095] M weights are selected from the Nin_i*Nout_i weights through the sliding window; when the M weights meet the preset condition, all or part of the M weights are set to zero, wherein M=Bin_i*Bout_i.

[1096] Furthermore, the first training adopts a back propagation algorithm, and the weights that have been set to zero during the training process remain zero.

[1097] Furthermore, the weights of the neural network are grouped in the following ways:

[1098] grouping the weights of the neural network into a group, and / or;

[1099] grouping the weights of the neural network by layer type, and / or;

[1100] Grouping of neural network weights between and / or within layers.

[1101] Furthermore, grouping the weights of the neural network according to layer type includes:

[1102] Divide the weights of all convolutional layers, all fully connected layers, and all LSTM layers in the neural network into one group.

[1103] Furthermore, the grouping of the weights of the neural network according to the layers includes:

[1104] The weights of one or more convolutional layers, the weights of one or more fully connected layers, and the weights of one or more long short-term memory network layers in the neural network are divided into one group.

[1105] Furthermore, grouping the weights of the neural network according to the layer includes:

[1106] The weights within a layer of the neural network are divided, and each part after division is divided into a group.

[1107] Furthermore, the clustering algorithm includes K-means, K-medoids, Clara and / or Clarans.

[1108] Furthermore, the center weight selection method is to minimize the cost function J(w, w0).

[1109] Furthermore, the cost function satisfies:

[1110]

[1111] Where w is the total weight of the class, w0 is the center weight, n is the number of weights in the class, wi is the i-th weight in the class, and i is an integer greater than 0 and less than or equal to n.

[1112] The second training of the clustered and encoded neural network includes: retraining the clustered and encoded neural network using the back propagation algorithm, the weights that have been set to 0 during the training process will always remain 0, and only the weight codebook is trained, not the weight dictionary.

[1113] A device for compressing neural network data, comprising:

[1114] A memory, for storing operation instructions;

[1115] The processor is used to execute the operation instruction in the memory, and operates according to any of the above-mentioned data compression methods when executing the operation instruction.

[1116] A processing device comprising:

[1117] a coarse-grained number selection unit, configured to input neuron and target weight position information and select neurons that require calculation, wherein the target weight is a weight whose absolute value is greater than a second preset threshold;

[1118] A lookup table unit is used to receive a quantized target weight dictionary and a target weight codebook, perform a table lookup operation, and obtain and output the target weight of the neural network;

[1119] The operation unit is used to receive the selected neurons and target weights, operate on the neural network to obtain and output neurons.

[1120] Furthermore, the lookup table unit is further configured to directly transmit the unquantized target weight to the operation unit via a bypass.

[1121] Furthermore, it also includes an instruction control unit for receiving instructions and decoding the instructions to obtain control information to control the operation unit.

[1122] Furthermore, it also includes a storage unit for storing neurons, weights and instructions of the neural network.

[1123] Furthermore, the storage unit is also used to store the target weight and the location information of the target weight; and is also used to store the quantized target weight codebook and the target weight dictionary.

[1124] Furthermore, the computing unit includes at least one of the following:

[1125] A multiplier, configured to multiply the first input data and the second input data to obtain multiplied data;

[1126] an addition tree, configured to add the third input data step by step through the addition tree, or to obtain added data by adding the third input data and the fourth input data;

[1127] The activation function operation unit obtains output data by performing an activation function operation on the fifth data, where the activation function is a sigmoid, tanh, relu or softmax function operation.

[1128] Furthermore, the operation unit also includes a pooling unit, which is used to obtain output data after the pooling operation by performing a pooling operation on the input sixth data, and the pooling operation includes: average pooling, maximum pooling or median pooling.

[1129] Furthermore, the processing device further includes:

[1130] The instruction control unit is used to receive instructions from the storage unit, generate control information after decoding to control the coarse-grained number selection unit to perform number selection operations, the lookup table unit to perform table lookup operations and the operation unit to perform calculation operations.

[1131] Furthermore, the instructions are neural network-specific instructions, including control instructions, data transmission instructions, operation instructions and logic instructions.

[1132] Furthermore, the neural network-specific instructions are the Cambricon instruction set, each instruction in the Cambircon instruction set is 64 bits long, and the instructions consist of an opcode and an operand.

[1133] Furthermore, the control instructions are used to control the execution process of the neural network, including jump instructions and conditional branch instructions.

[1134] Furthermore, data transfer instructions are used to complete data transfer between different storage media, including load instructions, store instructions, and move instructions.

[1135] Furthermore, the operation instructions are used to complete arithmetic operations of neural networks, and the operation instructions include matrix operation instructions, vector operation instructions, scalar operation instructions, convolutional neural network operation instructions, fully connected neural network operation instructions, pooling neural network operation instructions, RBM neural network operation instructions, LRN neural network operation instructions, LCN neural network operation instructions, LSTM neural network operation instructions, RNN neural network operation instructions, RELU neural network operation instructions, PRELU neural network operation instructions, SIGMOID neural network operation instructions, TANH neural network operation instructions and MAXOUT neural network operation instructions.

[1136] Furthermore, the logic instructions are used to complete the logic operations of the neural network, and the logic instructions include vector logic operation instructions and scalar logic operation instructions.

[1137] Furthermore, the vector logic operation instructions include vector comparison, vector logic operation and vector greater than merge instructions; preferably, vector comparison includes but is not limited to greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤) and not equal to; preferably, vector logic operations include and, or, and not.

[1138] Furthermore, the scalar logical operation includes scalar comparison and scalar logical operation; preferably, the scalar comparison includes but is not limited to greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤) and not equal to; preferably, the scalar logical operation includes logical AND, logical OR, and logical NOT.

[1139] Furthermore, the processing device further includes an instruction cache unit for caching instructions, and the instruction cache unit is an on-chip cache.

[1140] Furthermore, the processing device further includes a target weight codebook cache unit for caching the target weight codebook, and the target weight codebook cache unit is an on-chip cache.

[1141] Furthermore, the processing device further includes a target weight dictionary cache unit for caching the target weight dictionary, and the target weight dictionary cache unit is an on-chip cache.

[1142] Furthermore, the processing device also includes a target weight position cache unit for caching the target weight position and for mapping each connection weight in the input data to a corresponding input neuron one by one. The target weight position cache unit is an on-chip cache.

[1143] Furthermore, the target weight position cache unit is used to correspond each connection weight in the input data to the corresponding input neuron one-to-one, including: using 1 to indicate that the weight is connected to the input neuron, 0 to indicate no connection, and the connection status of each group of outputs and all inputs constitutes a string of 0 and 1 to represent the connection relationship of the output.

[1144] Furthermore, the target weight position cache unit is used to correspond each connection weight in the input data to the corresponding input neuron one-to-one, and the one-to-one correspondence includes: the distance between the input neuron position where the first connection of a group of outputs is located and the first input neuron, the distance between the second group of input neurons of the output and the previous input neuron, the distance between the third group of input neurons of the output and the previous input neuron, and so on, until all the inputs of the output are exhausted, to represent the connection relationship of the output.

[1145] Furthermore, the processing device further includes an input neuron cache unit for caching input neurons input to the coarse-grained number selection unit, and the input neuron cache unit is an on-chip cache.

[1146] Furthermore, the processing device further includes an output neuron cache unit for caching output neurons, and the output neuron cache unit is an on-chip cache.

[1147] Furthermore, the processing device also includes a direct data access unit DMA unit, which is used to read and write data or instructions in the storage unit, instruction cache unit, target weight codebook cache unit, target weight dictionary cache unit, target weight position cache unit, input neuron cache unit and output neuron cache unit.

[1148] Furthermore, the processing device further includes a preprocessing unit configured to preprocess the original data and input the preprocessed data into the storage unit.

[1149] A processing method comprising:

[1150] Input the neuron and target weight position information and select the neurons that need to be calculated;

[1151] Receive the quantized target weight dictionary and target weight codebook, perform table lookup operation and output the target weight of the neural network;

[1152] Receive the selected neurons and target weights, operate on the neural network and output neurons.

[1153] Furthermore, the processing method also includes: receiving unquantized target weights to perform neural network operations.

[1154] Furthermore, it also includes: receiving instructions, and generating control information after decoding to control the operation of the neural network.

[1155] Furthermore, the operation includes at least one of the following: a multiplication operation, multiplying the first input data and the second input data to obtain the multiplied data; an addition operation, adding the third input data step by step through an addition tree, or adding the third input data to the fourth input data to obtain the added data; an activation function operation, performing an activation function operation on the fifth data to obtain output data, and the activation function is a sigmoid, tanh, relu or softmax function operation.

[1156] Furthermore, the operation also includes a pooling operation, which is used to obtain output data after the pooling operation by performing a pooling operation on the input sixth data, and the pooling operation includes: average pooling, maximum pooling or median pooling.

[1157] Furthermore, the instructions are neural network-specific instructions, including control instructions, data transmission instructions, operation instructions and logic instructions.

[1158] Furthermore, the control instructions are used to control the execution process of the neural network, including jump instructions and conditional branch instructions.

[1159] Furthermore, data transfer instructions are used to complete data transfer between different storage media, including load instructions, store instructions, and move instructions.

[1160] Furthermore, the operation instructions are used to complete arithmetic operations of neural networks, including matrix operation instructions, vector operation instructions, scalar operation instructions, convolutional neural network operation instructions, fully connected neural network operation instructions, pooling neural network operation instructions, RBM neural network operation instructions, LRN neural network operation instructions, LCN neural network operation instructions, LSTM neural network operation instructions, RNN neural network operation instructions, RELU neural network operation instructions, PRELU neural network operation instructions, SIGMOID neural network operation instructions, TANH neural network operation instructions and MAXOUT neural network operation instructions.

[1161] Furthermore, the neural network-specific instructions are the Cambricon instruction set, and the instructions in the Cambircon instruction set are composed of opcodes and operands.

[1162] Each instruction in the Cambircon instruction set has a fixed length. For example, each instruction in the instruction set can be 64 bits long.

[1163] Furthermore, the logic instructions are used to complete the logic operations of the neural network, including vector logic operation instructions and scalar logic operation instructions.

[1164] Furthermore, the vector logic operation instructions include vector comparison, vector logic operation and vector greater than merge instructions; preferably, vector comparison includes but is not limited to greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤) and not equal to; preferably, vector logic operation includes logical AND, logical OR, or logical NOT.

[1165] Furthermore, the scalar logical operation includes scalar comparison and scalar logical operation; preferably, the scalar comparison includes but is not limited to greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤) and not equal to; preferably, the scalar logical operation includes logical AND, logical OR, and logical NOT.

[1166] Furthermore, the method further comprises the steps of: preprocessing the input neuron and target weight position information, wherein the preprocessing includes segmentation, Gaussian filtering, binarization, regularization and / or normalization.

[1167] Furthermore, after receiving the selected neurons and target weights, the processing method further includes the steps of: storing input neurons, weight dictionaries, codebooks and instructions, and storing output neurons; and caching the instructions, input neurons, and output neurons.

[1168] An electronic device comprising the processing device according to any of the preceding claims, wherein the electronic device comprises a data processing device, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a mobile phone, a driving recorder, a navigation system, a sensor, a camera, a cloud server, a camera, a video camera, a projector, a watch, a headset, a mobile storage device, a wearable device, a vehicle, a household appliance, and / or a medical device;

[1169] Among them, the transportation vehicles include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves and / or range hoods; the medical equipment includes magnetic resonance imaging (MRI), ultrasound machines and / or electrocardiographs.

[1170] A processing device, comprising:

[1171] A coarse-grained number selection unit is used to input neuron and target weight position information and select neurons that need to be calculated; wherein the target weight is a weight whose absolute value is greater than a preset threshold;

[1172] A lookup table unit is used to receive a quantized target weight dictionary and a target weight codebook, perform a table lookup operation, and obtain and output the target weight of the neural network;

[1173] The operation unit is used to receive the selected neurons and target weights, operate on the neural network to obtain and output neurons.

[1174] Furthermore, the lookup table unit is further configured to directly transmit the unquantized target weight to the operation unit via a bypass.

[1175] Furthermore, the processing device further includes:

[1176] The instruction control unit is used to receive instructions and decode the instructions to obtain control information to control the operation unit.

[1177] Furthermore, the processing device further includes:

[1178] Storage unit, used to store neurons, weights and instructions of the neural network.

[1179] Furthermore, the storage unit is also used to store the target weight and the location information of the target weight; and store the quantized target weight codebook and the target weight dictionary.

[1180] Furthermore, the computing unit includes at least one of the following:

[1181] A multiplier, configured to multiply the first input data and the second input data to obtain multiplied data;

[1182] an addition tree, configured to add the third input data step by step through the addition tree, or to obtain added data by adding the third input data and the fourth input data;

[1183] The activation function operation unit obtains output data by performing an activation function operation on the fifth data, where the activation function is a sigmoid, tanh, relu or softmax function operation.

[1184] Furthermore, the operation unit also includes a pooling unit, which is used to obtain output data after the pooling operation by performing a pooling operation on the input sixth data, and the pooling operation includes: average pooling, maximum pooling or median pooling.

[1185] Furthermore, the processing device further includes:

[1186] The instruction control unit is used to receive instructions from the storage unit, decode the instructions and generate control information to control the coarse-grained number selection unit to perform number selection operations, the lookup table unit to perform table lookup operations and the operation unit to perform calculation operations.

[1187] Furthermore, the instructions are neural network-specific instructions, including control instructions, data transmission instructions, operation instructions and logic instructions.

[1188] Furthermore, the neural network-specific instructions are the Cambricon instruction set.

[1189] Furthermore, the processing device further includes:

[1190] The instruction cache unit is used to cache instructions, and the instruction cache unit is an on-chip cache.

[1191] Furthermore, the processing device further includes:

[1192] The target weight codebook cache unit is used to cache the target weight codebook, and the target weight codebook cache unit is an on-chip cache.

[1193] Furthermore, the processing device further includes:

[1194] The target weight dictionary cache unit is used to cache the target weight dictionary, and the target weight dictionary cache unit is an on-chip cache.

[1195] Furthermore, the processing device further includes:

[1196] The target weight position cache unit is used to cache the target weight position and is also used to correspond each connection weight in the input data to the corresponding input neuron one by one. The target weight position cache unit is an on-chip cache.

[1197] Furthermore, the target weight position cache unit is used to correspond each connection weight in the input data to the corresponding input neuron one-to-one, including:

[1198] 1 is used to indicate that the weight is connected to the input neuron, and 0 is used to indicate that there is no connection. The connection status of each set of outputs and all inputs forms a string of 0 and 1 to represent the connection relationship of the output.

[1199] Furthermore, the target weight position cache unit is used to correspond each connection weight in the input data to the corresponding input neuron one-to-one, including:

[1200] The connection relationship of a group of outputs is represented by the distance from the input neuron where the first connection of the output is located to the first input neuron, the distance from the second group of input neurons of the output to the previous input neuron, the distance from the third group of input neurons of the output to the previous input neuron, and so on, until all the inputs of the output are exhausted.

[1201] Furthermore, the processing device further includes:

[1202] The input neuron cache unit is used to cache the input neurons input to the coarse-grained number selection unit, and the input neuron cache unit is an on-chip cache.

[1203] Furthermore, the processing device further includes:

[1204] The output neuron cache unit is used to cache output neurons, and the output neuron cache unit is an on-chip cache.

[1205] Furthermore, the processing device further includes:

[1206] A direct data access DMA unit is used to read and write data or instructions in the storage unit, instruction cache unit, target weight codebook cache unit, target weight dictionary cache unit, target weight position cache unit, input neuron cache unit and output neuron cache unit.

[1207] Furthermore, the processing device further includes:

[1208] Preprocessing unit: used for preprocessing the original data and inputting the preprocessed data into the storage unit.

[1209] A processing method comprising:

[1210] Input neuron and target weight position information, select neurons that need to be calculated, and the target weight is a weight whose absolute value is greater than a preset threshold;

[1211] Receive the quantized target weight dictionary and target weight codebook, perform table lookup operation to obtain and output the neural network target weight;

[1212] Receive the selected neurons and target weights, operate on the neural network to obtain and output neurons.

[1213] Furthermore, the processing method further includes:

[1214] Receives unquantized target weights for neural network operations.

[1215] Furthermore, the method further comprises:

[1216] Receive instructions and decode the instructions to obtain control information for controlling the operation of the neural network.

[1217] Furthermore, the operation includes at least one of the following:

[1218] A multiplication operation, multiplying the first input data and the second input data to obtain multiplied data;

[1219] An addition operation, adding the third input data level by level through an addition tree, or adding the third input data to the fourth input data to obtain added data;

[1220] An activation function operation is performed on the fifth data to obtain output data, where the activation function is a sigmoid, tanh, relu or softmax function operation.

[1221] Furthermore, the operation also includes:

[1222] A pooling operation is used to obtain output data after the pooling operation by performing a pooling operation on the input sixth data, wherein the pooling operation includes: average pooling, maximum pooling or median pooling.

[1223] Furthermore, the instructions are neural network-specific instructions, including control instructions, data transmission instructions, operation instructions and logic instructions.

[1224] Furthermore, the neural network-specific instructions are the Cambricon instruction set, each instruction in the Cambircon instruction set is 64 bits long, and the instructions consist of an opcode and an operand.

[1225] Furthermore, the method further comprises the steps of:

[1226] The input neuron and target weight position information is preprocessed, and the preprocessing includes segmentation, Gaussian filtering, binarization, regularization and / or normalization.

[1227] Furthermore, after receiving the selected neuron and the target weight, the method further includes the steps of:

[1228] Storing the input neurons, weight dictionary, codebook and instructions, and storing the output neurons; and caching the instructions, input neurons and output neurons.

[1229] A data compression method, comprising:

[1230] Performing coarse-grained pruning of the weights of the neural network, including: selecting M weights from the neural network according to a sliding window, and setting all or part of the M weights to zero when the M weights meet a preset condition; performing a first training on the neural network, and maintaining the weights that have been set to zero during the training process at zero;

[1231] The weights of the neural network are quantified, including: grouping the weights of the neural network, clustering each group of weights according to a clustering algorithm, calculating the central weight of each class, and replacing all weights in each class with the central weight corresponding to the class; encoding the central weight to obtain a code book and a weight dictionary, and performing a second training on the neural network. During the retraining, only the code book is trained, and the content of the weight dictionary remains unchanged.

[1232] Furthermore, the preset conditions are:

[1233] The amount of information of the M weights is less than a first preset threshold.

[1234] Furthermore, the information amount of the M weights is the arithmetic mean of the absolute values ​​of the M weights, the geometric mean of the absolute values ​​of the M weights, or the maximum value of the M weights, the first preset threshold is the first threshold, the second threshold, or the third threshold, and the information amount of the M weights is less than the first preset threshold, including:

[1235] The arithmetic mean of the absolute values ​​of the M weights is less than the first threshold, or the geometric mean of the absolute values ​​of the M weights is less than the second threshold, or the maximum value of the M weights is less than the third threshold.

[1236] Furthermore, the processing method further includes: repeatedly selecting M weights from the neural network using a sliding window, and when the M weights meet a preset condition, setting all or part of the M weights to zero; and performing a first retraining on the neural network until no weights can be set to zero without losing a set accuracy. The set accuracy is x%, where x is between 0 and 5.

[1237] Furthermore, the neural network includes a fully connected layer, a convolutional layer, and an LSTM layer; M weights are selected from the neural network according to the sliding window, including:

[1238] The weights of the fully connected layer are a two-dimensional matrix (Nin, Nout), where Nin is the number of input neurons and Nout is the number of output neurons. The fully connected layer has Nin*Nout weights, and the size of the sliding window is a sliding window of Bin*Bout, where Bin is an integer greater than 0 and less than or equal to Nin, and Bout is an integer greater than 0 and less than or equal to Nout. The processing device performs coarse-grained pruning on the weights of the fully connected layer of the neural network, including:

[1239] The sliding window is enabled to slide along the direction of Bin according to a step size Sin, or to slide along the direction of Bout according to a step size Sout, where Sin is a positive integer greater than 0 and less than or equal to Bin, and Sout is a positive integer greater than 0 and less than or equal to Bout;

[1240] Selecting M values ​​from the Nin*Nout weights through the sliding window, where M=Bin*Bout;

[1241] The processing device selects M weights from the convolutional layer of the neural network, including:

[1242] The weight of the convolutional layer is a four-dimensional matrix (Nfin, Nfout, Kx, Ky), where Nfin is the number of input feature images, Nfout is the number of output feature images, (Kx, Ky) is the size of the convolution kernel, and the convolutional layer has Nfin*Nfout*Kx*Ky weights; the sliding window is a four-dimensional sliding window of size Bfin*Bfout*Bx*By, where Bfin is an integer greater than 0 and less than or equal to Nfin, Bfout is an integer greater than 0 and less than or equal to Nfout, Bx is an integer greater than 0 and less than or equal to Kx, and By is an integer greater than 0 and less than or equal to Ky;

[1243] Slide the sliding window along the direction of Bfin with a step size Sfin, or slide along the direction of Bfout with a step size Sfout, or slide along the direction of Bx with a step size S, or slide along the direction of By with a step size Sy, where Sfin is an integer greater than 0 and less than or equal to Bfin, Sfout is an integer greater than 0 and less than or equal to Bfout, Sx is an integer greater than 0 and less than or equal to Bx, and Sy is an integer greater than 0 and less than or equal to By;

[1244] M weights are selected from the Nfin*Nfout*Kx*Ky weights through the sliding window, where M=Bfin*Bfout*Bx*By.

[1245] The processing device selects M weights from the LSTM layer of the neural network, including:

[1246] The weight of the LSTM layer is composed of the weights of m fully connected layers, where m is an integer greater than 0, and the weight of the i-th fully connected layer is (Nin_i, Nout_i), where i is an integer greater than 0 and less than or equal to m, Nin_i represents the number of input neurons of the i-th fully connected layer weight, and Nout_i represents the number of output neurons of the i-th fully connected layer weight; the size of the sliding window is Bin_i*Bout_i, where Bin_i is an integer greater than 0 and less than or equal to Nin_i, and Bout_i is an integer greater than 0 and less than or equal to Nout_i;

[1247] Slide the sliding window along the direction of Bin_i according to the step size Sin_i, or slide along the direction of Bout_i according to the step size Sout_i, where Sin_i is a positive integer greater than 0 and less than or equal to Bin_i, and Sout_i is a positive integer greater than 0 and less than or equal to Bout_i;

[1248] M weights are selected from the Bin_i*Bout_i weights through the sliding window, where M=Bin_i*Bout_i.

[1249] Furthermore, the first training adopts a back propagation algorithm, and the weights that have been set to zero during the training process remain zero.

[1250] Furthermore, the weights of the neural network are grouped in the following ways:

[1251] grouping the weights of the neural network into a group, and / or;

[1252] grouping the weights of the neural network by layer type, and / or;

[1253] Grouping of neural network weights between and / or within layers.

[1254] Furthermore, grouping the weights of the neural network according to layer type includes:

[1255] Divide the weights of all convolutional layers, all fully connected layers, and all LSTM layers in the neural network into one group.

[1256] Furthermore, the grouping of the weights of the neural network according to the layers includes:

[1257] The weights of one or more convolutional layers, the weights of one or more fully connected layers, and the weights of one or more long short-term memory network layers in the neural network are divided into one group.

[1258] Furthermore, grouping the weights of the neural network according to the layer includes:

[1259] The weights within a layer of the neural network are divided, and each part after division is divided into a group.

[1260] Furthermore, the clustering algorithm includes K-means, K-medoids, Clara and / or Clarans.

[1261] Furthermore, the center weight selection method is to minimize the cost function J(w,w0),

[1262]

[1263] Where w is the total weight of the class, w0 is the center weight, n is the number of weights in the class, wi is the i-th weight in the class, and i is an integer greater than 0 and less than or equal to n.

[1264] The second training of the clustered and encoded neural network includes: retraining the clustered and encoded neural network using the back propagation algorithm, the weights that have been set to 0 during the training process will always remain 0, and only the weight codebook is trained, not the weight dictionary.

[1265] A device for compressing neural network data, comprising:

[1266] A memory, for storing operation instructions;

[1267] The processor is used to execute the operation instruction in the memory, and operates according to any of the compression methods described above when executing the operation instruction.

[1268] A processing device, comprising:

[1269] a coarse-grained number selection unit, configured to input neuron and target weight position information and select neurons that require calculation, wherein the target weight is a weight whose absolute value is greater than a second preset threshold;

[1270] A lookup table unit is used to receive a quantized target weight dictionary and a target weight codebook, perform a table lookup operation, and obtain and output the target weight of the neural network;

[1271] The operation unit is used to receive the selected neurons and target weights, operate on the neural network to obtain and output neurons.

[1272] Furthermore, the lookup table unit is further configured to directly transmit the unquantized target weight to the operation unit via a bypass.

[1273] Furthermore, the processing device also includes an instruction control unit for receiving instructions and decoding the instructions to obtain control information to control the operation unit.

[1274] Furthermore, the processing device also includes a storage unit for storing neurons, weights and instructions of the neural network.

[1275] Furthermore, the storage unit is also used to store the target weight and the location information of the target weight; and is also used to store the quantized target weight codebook and the target weight dictionary.

[1276] Furthermore, the computing unit includes at least one of the following:

[1277] A multiplier, configured to multiply the first input data and the second input data to obtain multiplied data;

[1278] an addition tree, configured to add the third input data step by step through the addition tree, or to obtain added data by adding the third input data and the fourth input data;

[1279] The activation function operation unit obtains output data by performing an activation function operation on the fifth data, where the activation function is a sigmoid, tanh, relu or softmax function operation.

[1280] Furthermore, the operation unit also includes a pooling unit, which is used to obtain output data after the pooling operation by performing a pooling operation on the input sixth data, and the pooling operation includes: average pooling, maximum pooling or median pooling.

[1281] Furthermore, the processing device also includes an instruction control unit for receiving instructions from the storage device, decoding the instructions to generate control information to control the coarse-grained number selection unit to perform number selection operations, the lookup table unit to perform table lookup operations and the operation unit to perform calculation operations.

[1282] Furthermore, the instructions are neural network-specific instructions, including control instructions, data transmission instructions, operation instructions and logic instructions.

[1283] Furthermore, the neural network-specific instructions are the Cambricon instruction set, each instruction in the Cambircon instruction set is 64 bits long, and the instructions consist of an opcode and an operand.

[1284] Furthermore, the control instructions are used to control the execution process of the neural network, including jump instructions and conditional branch instructions.

[1285] Furthermore, data transfer instructions are used to complete data transfer between different storage media, including load instructions, store instructions, and move instructions.

[1286] Furthermore, the operation instructions are used to complete arithmetic operations of neural networks, including matrix operation instructions, vector operation instructions, scalar operation instructions, convolutional neural network operation instructions, fully connected neural network operation instructions, pooling neural network operation instructions, RBM neural network operation instructions, LRN neural network operation instructions, LCN neural network operation instructions, LSTM neural network operation instructions, RNN neural network operation instructions, RELU neural network operation instructions, PRELU neural network operation instructions, SIGMOID neural network operation instructions, TANH neural network operation instructions and MAXOUT neural network operation instructions.

[1287] Furthermore, the logic instructions are used to complete the logic operations of the neural network, including vector logic operation instructions and scalar logic operation instructions.

[1288] Furthermore, the vector logic operation instructions include vector comparison, vector logic operation and vector greater than merge instructions; preferably, vector comparison includes but is not limited to greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤) and not equal to; preferably, vector logic operations include and, or, and not.

[1289] Furthermore, the scalar logical operation includes scalar comparison and scalar logical operation; preferably, the scalar comparison includes but is not limited to greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤) and not equal to; preferably, the scalar logical operation includes logical AND, logical OR, and logical NOT.

[1290] Furthermore, the processing device further includes an instruction cache unit for caching instructions, and the instruction cache unit is an on-chip cache.

[1291] Furthermore, the processing device further includes a target weight codebook cache unit for caching the target weight codebook, and the target weight codebook cache unit is an on-chip cache.

[1292] Furthermore, the processing device further includes a target weight dictionary cache unit for caching the target weight dictionary, and the target weight dictionary cache unit is an on-chip cache.

[1293] Furthermore, the processing device also includes a target weight position cache unit for caching the target weight position and for mapping each connection weight in the input data to a corresponding input neuron one by one. The target weight position cache unit is an on-chip cache.

[1294] Furthermore, the target weight position cache unit is used to correspond each connection weight in the input data to the corresponding input neuron one-to-one, including: using 1 to indicate that the weight is connected to the input neuron, 0 to indicate no connection, and the connection status of each group of outputs and all inputs constitutes a string of 0 and 1 to represent the connection relationship of the output.

[1295] Furthermore, the target weight position cache unit is used to correspond each connection weight in the input data to the corresponding input neuron one-to-one, and the one-to-one correspondence includes: the distance between the input neuron position where the first connection of a group of outputs is located and the first input neuron, the distance between the second group of input neurons of the output and the previous input neuron, the distance between the third group of input neurons of the output and the previous input neuron, and so on, until all the inputs of the output are exhausted, to represent the connection relationship of the output.

[1296] Furthermore, the processing device further includes an input neuron cache unit for caching input neurons input to the coarse-grained number selection unit, and the input neuron cache unit is an on-chip cache.

[1297] Furthermore, the processing device further includes an output neuron cache unit for caching output neurons, and the output neuron cache unit is an on-chip cache.

[1298] Furthermore, the processing device also includes a direct data access DMA unit for reading and writing data or instructions in the storage unit, instruction cache unit, target weight codebook cache unit, target weight dictionary cache unit, target weight position cache unit, input neuron cache unit and output neuron cache unit.

[1299] Furthermore, the processing device further includes a preprocessing unit configured to preprocess the original data and input the preprocessed data into the storage unit.

[1300] A processing method comprising:

[1301] Input neuron and target weight position information, select neurons that need to be calculated, and the target weight is a weight whose absolute value is greater than a preset threshold;

[1302] Receive the quantized target weight dictionary and target weight codebook, perform table lookup operation to obtain and output the neural network target weight;

[1303] Receive the selected neurons and target weights, operate on the neural network to obtain and output neurons.

[1304] Furthermore, the processing method also includes: receiving unquantized target weights to perform neural network operations.

[1305] Furthermore, it also includes: receiving instructions and decoding the instructions to obtain control information for controlling the operation of the neural network.

[1306] Furthermore, the operation includes at least one of the following: a multiplication operation, multiplying the first input data and the second input data to obtain the multiplied data; an addition operation, adding the third input data step by step through an addition tree, or adding the third input data to the fourth input data to obtain the added data; an activation function operation, performing an activation function operation on the fifth data to obtain output data, and the activation function is a sigmoid, tanh, relu or softmax function operation.

[1307] Furthermore, the operation also includes a pooling operation, which is used to obtain output data after the pooling operation by performing a pooling operation on the input sixth data, and the pooling operation includes: average pooling, maximum pooling or median pooling.

[1308] Furthermore, the instructions are neural network-specific instructions, including control instructions, data transmission instructions, operation instructions and logic instructions.

[1309] Furthermore, the control instructions are used to control the execution process of the neural network, including jump instructions and conditional branch instructions.

[1310] Furthermore, data transfer instructions are used to complete data transfer between different storage media, including load instructions, store instructions, and move instructions.

[1311] Furthermore, the operation instructions are used to complete arithmetic operations of neural networks, including matrix operation instructions, vector operation instructions, scalar operation instructions, convolutional neural network operation instructions, fully connected neural network operation instructions, pooling neural network operation instructions, RBM neural network operation instructions, LRN neural network operation instructions, LCN neural network operation instructions, LSTM neural network operation instructions, RNN neural network operation instructions, RELU neural network operation instructions, PRELU neural network operation instructions, SIGMOID neural network operation instructions, TANH neural network operation instructions and MAXOUT neural network operation instructions.

[1312] Furthermore, the neural network-specific instructions are the Cambricon instruction set, each instruction in the Cambircon instruction set is 64 bits long, and the instructions consist of an opcode and an operand.

[1313] Furthermore, the logic instructions are used to complete the logic operations of the neural network, including vector logic operation instructions and scalar logic operation instructions.

[1314] Furthermore, the vector logic operation instructions include vector comparison, vector logic operation and vector greater than merge instructions; preferably, vector comparison includes but is not limited to greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤) and not equal to; preferably, vector logic operation includes logical AND, logical OR, or logical NOT.

[1315] Furthermore, the scalar logical operation includes scalar comparison and scalar logical operation; preferably, the scalar comparison includes but is not limited to greater than, less than, equal to, greater than or equal to (≥), less than or equal to (≤) and not equal to; preferably, the scalar logical operation includes logical AND, logical OR, and logical NOT.

[1316] Furthermore, the method further comprises the steps of: preprocessing the input neuron and target weight position information, wherein the preprocessing includes segmentation, Gaussian filtering, binarization, regularization and / or normalization.

[1317] Furthermore, after receiving the selected neurons and target weights, the method further includes the steps of storing input neurons, weight dictionaries, codebooks and instructions, and storing output neurons; and caching the instructions, input neurons and output neurons.

[1318] In aspect 34, an embodiment of the present invention provides an electronic device, comprising the processing device according to any of the preceding claims, wherein the electronic device includes a data processing device, a robot, a computer, a printer, a scanner, a tablet computer, a smart terminal, a mobile phone, a driving recorder, a navigation system, a sensor, a camera, a cloud server, a camera, a video camera, a projector, a watch, a headset, a mobile storage device, a wearable device, a vehicle, a household appliance, and / or a medical device;

[1319] The transportation means include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves and / or range hoods; the medical equipment include magnetic resonance imaging machines, ultrasound machines and / or electrocardiographs.

[1320] A computing device, comprising:

[1321] a screening unit (400) for screening out a feature map and its corresponding weights based on a connection state array of a feature map composed of output neurons and input neurons, and outputting the feature values ​​and the corresponding weights to the operation unit (600), and / or;

[1322] for filtering out feature map rows and their corresponding weight rows based on a connection state array of each row in a feature map composed of output neurons and input neurons, and outputting the feature map rows and their corresponding weight rows to the operation unit (600), and / or;

[1323] for filtering out a feature map column and its corresponding weight column according to a connection state array of each column in a feature map composed of output neurons and input neurons, and outputting the feature map column and its corresponding weight column to the operation unit (600); and

[1324] The operation unit (600) performs corresponding artificial neural network operations supporting structure clipping on the data output by the screening unit (400) according to the instructions to obtain output neurons.

[1325] Furthermore, the screening process of the screening unit (400) includes:

[1326] If the weights have not been screened offline, the feature graph and its corresponding weights are screened out according to the connection state array of the feature graph composed of the output neurons and the input neurons, and then the screened feature graph and its corresponding weights are output to the operation unit, and / or; the feature graph rows / columns and their corresponding weight rows / columns are screened out according to the connection state array of the rows / columns in the feature graph composed of the output neurons and the input neurons, and then the screened feature graph rows / columns and their corresponding weight rows / columns are output to the operation unit;

[1327] If the weights have been filtered offline, the feature map is filtered out according to the connection state array of the feature map composed of the output neurons and the input neurons, and then the filtered feature map is output to the operation unit, and the filtered weights are directly transmitted to the operation unit without passing through the filtering unit; and / or the feature map rows / columns and their corresponding weight rows / columns are filtered out according to the connection state array of the rows / columns in the feature map composed of the output neurons and the input neurons, and the filtered feature map rows / columns and their corresponding weight rows / columns are output to the operation unit.

[1328] Furthermore, the connection state array is used to represent the connection state between the output neuron and the input neuron, and the expression includes:

[1329] The first way of expression: the numbers "0" and "1" are used to represent the connection status, where "1" represents connection and "0" represents no connection; or "0" represents connection and "1" represents no connection; for the connection status array of the feature map composed of output neurons and input neurons, the connection status of each output neuron and the feature map composed of all input neurons forms an array including "0" and "1"; for the connection status array of rows / columns in the feature map composed of output neurons and input neurons, the connection status of each output neuron and the row / column in the feature map composed of all input neurons forms an array including "0" and "1";

[1330] The second way of expression: use distance to represent the connection state. For the connection state array of the feature graph composed of output neurons and input neurons, the distance from the input neuron position of the first connection of each output neuron to the first input neuron, the distance from the input neuron position of the second connection to the input neuron where the previous connection is located, the distance from the input neuron position of the third connection to the input neuron where the previous connection is located, and so on, until all input neurons connected to the output neuron are exhausted and combined into an array to represent the connection state array corresponding to the output neuron.

[1331] Furthermore, the elements in the array of the second expression are non-negative integers.

[1332] Furthermore, the feature graph and the corresponding weights, or the feature graph rows / columns and the corresponding weight rows / columns, which are screened using the connection state array of the feature graph composed of the output neurons and the input neurons or the connection state array of the rows / columns in the feature graph composed of the output neurons and the input neurons, satisfy the following conditions:

[1333] The first expression is: the feature graphs and weights corresponding to the output neurons that are filtered out for calculation, and the corresponding connection state arrays, where the element values ​​are all numbers indicating that there are connections; and / or the feature graph rows / columns and weight rows / columns that are filtered out for calculation of the output neurons, and the corresponding connection state arrays, where the element values ​​are all numbers indicating that there are connections;

[1334] The second expression: for each output neuron O j The corresponding connection state array Index A, each value is a non-negative integer, and all input neurons are I1, I2, ..., I N , let the input neuron connected to the output neuron be n≤N,k1,k2,...,k n ∈{1,2,...,N}, their values ​​are unknown, then Index A has n bits, the value of the first bit is A1, which indicates the distance from the input neuron position of the first connection to I1; the value of the pth bit is A P , p=2,3,...,n, represents the distance between the input neuron position of the current connection and the input neuron position of the previous connection; then the selected ones are used to calculate the output neuron O j The input neuron is: The corresponding weight is Satisfies: k1=A1+1, k p =A p +k p-1 .

[1335] Furthermore, the computing device further comprises: a storage unit (100) for storing data and instructions;

[1336] The data stored in the storage unit (100) includes:

[1337] The feature map composed of input neurons, weights, connection state array of the feature map composed of output neurons and input neurons, connection state array of each row in the feature map composed of output neurons and input neurons, connection state array of each column in the feature map composed of output neurons and input neurons, feature map composed of output neurons, and other data and parameters required for artificial neural network operations.

[1338] Furthermore, the computing unit (600) includes:

[1339] A first part multiplier performs a multiplication operation on the first input data and the second input data to obtain output data;

[1340] The second part consists of one or more adders, which add the multiplication results step by step through the adder tree to obtain a weighted sum, and bias or do no processing on the weighted sum as needed;

[1341] The third part is an activation function unit, which performs an activation function operation on the weighted sum result to obtain an output neuron.

[1342] Furthermore, the one or more adders form an adder tree.

[1343] Furthermore, the operation unit (600) further includes: a pooling unit that performs a pooling operation on the input data.

[1344] Furthermore, the computing device further comprises: a cache unit (300) which caches data and instructions.

[1345] A device for performing forward operation of an artificial neural network, comprising a floating-point data statistics module, a floating-point data conversion unit and a floating-point data operation module,

[1346] The floating-point data statistics module is used to perform statistical analysis on various types of data required for the forward operation of the artificial neural network to obtain the exponent bit offset offset and the exponent bit length EL;

[1347] The floating-point data conversion unit is used to realize the conversion of the long-digit floating-point data type to the short-digit floating-point data type according to the exponent bit offset offset and the exponent bit length EL obtained by the floating-point data statistics module;

[1348] The floating-point data operation...

Claims

1. An information processing method, characterized in that: Applied to a terminal device provided with a computing device, wherein the computing device stores an instruction set including at least one operation instruction, the method comprises: Acquire first information, where the first information is information to be processed by the terminal device; Invoking a computing instruction in the computing device to process the first information, thereby obtaining second information; Output the second information The computing device comprises: a storage medium, a register unit, an interconnection module, an operation unit, a controller unit and a data access unit; An interconnection module, used for controlling the connection relationship between the calculators in the computing unit so that at least two calculators form different computing topologies; A register unit, used to store operation instructions, addresses of data blocks in storage media, and computing topology structures corresponding to the operation instructions; A controller unit is used to extract a computing instruction, an operation domain corresponding to the computing instruction, and a first computing topology structure corresponding to the computing instruction from the register unit, decode the computing instruction into an execution instruction, the execution instruction is used to control the computing unit to perform the computing operation, transmit the operation domain to the data access unit, and transmit the computing topology structure to the interconnection module; A data access unit, used for extracting a data block corresponding to the operation domain from a storage medium and transmitting the data block to a computing unit; An interconnection module, used for forming a first computing topology structure according to the connection relationship of the calculators in the control computing unit; The computing unit is used to call the calculator to perform computing operations on the data block according to the first computing topology structure and the execution instruction to obtain computing results, and transmit the computing results to the data access unit for storage in the storage medium.

2. The method according to claim 1, characterized in that The obtaining of the first information comprises: The obtained original information is preprocessed to obtain the first information, where the first information is information with a preset format, and the preprocessing includes at least one of the following: data deduplication processing, data encoding processing, data conversion processing, and normalization processing.

3. The method according to claim 1, characterized in that When the first information is voice information, calling a calculation instruction in the computing device to process the first information to obtain the second information includes: Invoking a speech recognition algorithm in a computing device to perform speech recognition on the speech information, thereby obtaining second information; The second information is text information, and the speech recognition algorithm includes at least one operation instruction for speech recognition.

4. The method according to claim 1, characterized in that When the first information is image information, calling a computing instruction in the computing device to process the first information to obtain second information includes: Invoking an image style conversion algorithm in a computing device to convert the style of the image information, thereby obtaining second information; The second information has a different style from the first information, and the image style conversion algorithm includes at least one operation instruction for image style or image conversion.

5. The method according to claim 1, characterized in that When the first information is image information, and the image information includes at least one object to be identified, calling a computing instruction in the computing device to process the first information to obtain the second information includes: An object detection algorithm is called in a computing device to perform object detection on the image information, thereby obtaining second information; wherein the second information at least includes the object position, and the object detection algorithm includes at least one operation instruction for object detection.

6. The method according to claim 1, characterized in that When the first information is language information to be translated, calling a calculation instruction in the computing device to process the first information to obtain second information includes: Invoking a language translation algorithm in a computing device to translate the language information, thereby obtaining second information; The first information and the second information are different, and the language translation algorithm includes at least one operation instruction for language translation.

7. A terminal device, characterized in that: The method comprises a functional unit for executing the method as claimed in any one of claims 1 to 6.

8. A terminal device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images