Neural network computing method and neural network computing device

Abstract

The application provides a neural network computing method, comprising: receiving first input data; converting the first input data into second input data according to a replication relationship, wherein the second input data comprises the first input data and replicated data of at least part of the data in the first input data, and the replication relationship is used to determine the number of the replicated data of the at least part of the data; and performing neural network computation on the second input data based on target weights of a neural network set in a computing array. The neural network computing method provided by the application can improve the performance of the neural network under the premise that the unit precision of the computing array is limited.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a neural network computing method and a neural network computing device. Background Technology

[0002] Neural networks are tools for realizing artificial intelligence, characterized by high computational demands and memory-intensive processing of input data. One method to improve the processing efficiency of neural networks is to deploy them using an in-memory computing architecture. This method leverages the characteristic that weights remain constant during computation, pre-writing the weights into the computing storage medium and simultaneously performing weight storage and computation, thereby reducing the time and energy consumption associated with data exchange and computation.

[0003] In in-memory computing architectures, the crossbar (XB) is the core module for implementing storage and computation. An XB is typically a multiply-accumulate array built from non-volatile storage media, containing several rows and columns. Rows can also be called word lines, and columns can be called bit lines. The intersections of rows and columns are the computational storage units of the XB, which can be simply referred to as cells or computation units. During neural network computation, weights are pre-stored in the cells in the form of electrical conductance. When input data enters the XB in the form of voltage, the voltage flows through the corresponding conductance to form currents that converge in the same column. The sum of these currents can be used to represent the accumulated result of the product of the input data and the weights.

[0004] The computational units that make up an XB network typically have precision limitations, meaning that the number of bits that a computational unit can store for weights is limited. Low bit weights can lead to a decrease in the performance of the neural network. How to improve the performance of the neural network under the premise of limited precision of the computational units is a problem that needs to be solved. Summary of the Invention

[0005] This application provides a neural network computing method and a neural network computing device, which can improve the performance of neural networks under the premise that the accuracy of computing units in neural networks is limited.

[0006] In a first aspect, a neural network computation method is provided, comprising: receiving first input data; converting the first input data into second input data according to a replication relationship, wherein the second input data includes the first input data and replication data of at least a portion of the first input data, the replication relationship being used to determine the number of replication data of the at least a portion of the data; and performing neural network computation on the second input data based on the target weights of the neural network set in the computation array.

[0007] Due to limitations in device manufacturing processes, the data range that a computing array's computing units can represent is finite; that is, the unit precision of the computing array is limited. To deploy larger weights on a computing array with relatively low unit precision, the original weights need to be split to obtain the target weights and replication relationships. Applying the neural network computation method provided in this application, during neural network computation, the number of copies of the first input data can be determined based on the replication relationship. The first input data is then converted into second input data and used in neural network computation with the target weights. This enables neural network computation with large weights on a computing array with relatively low unit precision, improving the performance of the neural network.

[0008] Optionally, the replication relationship is determined based on the data representation range to which the quantized initial weight values ​​corresponding to the at least part of the data belong and the data representation range supported by the computing array.

[0009] When the data representation range of the quantized initial weight value does not exceed the data representation range supported by the computing array, there is no need to split the quantized initial weight value, that is, the number of copies is 0. When the data representation range of the quantized initial weight value exceeds the data representation range supported by the computing array, the quantized initial weight value needs to be split. At this time, the number of copies can be determined according to the data representation range supported by the computing array, so that each weight value obtained by splitting falls within the data representation range supported by the computing array. Therefore, the replication relationship obtained in this embodiment enables the number of copies of the input data to meet the computing requirements of the computing array.

[0010] Optionally, the method further includes: quantizing the initial weight values ​​in the initial weights of the neural network to obtain quantized initial weights, wherein at least a portion of the quantized initial weight values ​​do not belong to the data representation range supported by the computing array; determining the target weights based on the quantized initial weights, wherein the target weights include the result of splitting at least a portion of the quantized initial weight values, and any one of the target weight values ​​belongs to the data representation range supported by the computing array.

[0011] When the data representation range of the quantized initial weight value does not exceed the data representation range supported by the computing array, it is not necessary to split the quantized initial weight value, that is, the number of copies is 0. When the data representation range of the quantized initial weight value exceeds the data representation range supported by the computing array, it is necessary to split the quantized initial weight value. In this case, the number of copies can be determined according to the data representation range supported by the computing array, so that each weight value obtained by splitting falls within the data representation range supported by the computing array. Therefore, the target weight obtained in this embodiment can meet the computing requirements of the computing array.

[0012] Optionally, the quantized initial weight value includes a first initial weight value, and determining the target weight based on the quantized initial weight includes: splitting the first initial weight value into N target weight values, wherein the N target weight values ​​all belong to the data representation range supported by the computing array, the sum of the N target weight values ​​is less than or equal to the first initial weight value, and N is an integer greater than 1.

[0013] When the initial weight value is within the data representation range supported by N units, the N target weight values ​​obtained from the split are equal to the initial weight value. When the initial weight value exceeds the data representation range supported by N units, since N is the maximum number of units that can represent the initial weight value, the N target weight values ​​obtained from the split are less than the initial weight value. This embodiment avoids splitting failure caused by the target weight value exceeding the data representation range supported by N units.

[0014] Optionally, the number of rows of the target weight is less than or equal to the total number of rows of the computation array.

[0015] This embodiment can prevent the target weight from occupying more rows than the total number of rows in the computing array, which would cause weight deployment failure.

[0016] In a second aspect, a neural network computing device is provided, including a unit for performing any of the methods in the first aspect.

[0017] Thirdly, another neural network computing device is provided, including a processor and a memory for storing a computer program, and the processor for calling and running the computer program from the memory, such that the device performs any of the methods in the first aspect.

[0018] Fourthly, a computer program product is provided, the computer program product comprising: computer program code, which, when run by a device for training an intelligent agent, causes the device to perform any of the methods in the first aspect.

[0019] Fifthly, a computer-readable medium is provided that stores program code, the program code including instructions for performing any of the methods in the first aspect. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of a computing array provided in this application;

[0021] Figure 2 This is a schematic diagram illustrating the impact of unit precision on neural network computation;

[0022] Figure 3 This is a schematic diagram illustrating the impact of a trans-impedance amplifier (TIA) on neural network computation.

[0023] Figure 4 This is a schematic diagram of a weight splitting method provided in this application;

[0024] Figure 5 This is a schematic diagram of a neural network calculation method provided in this application;

[0025] Figure 6 This is a schematic diagram illustrating the impact of two different weights on the computational results of a neural network.

[0026] Figure 7 This is a schematic diagram of a neural network processing device provided in this application;

[0027] Figure 8 This is a schematic diagram of another neural network processing device provided in this application. Detailed Implementation

[0028] To facilitate understanding of the technical solution of this application, the concepts involved in this application will be briefly introduced first.

[0029] Artificial neural networks (ANNs), also known simply as neural networks (NNs) or neural network-like systems, are mathematical or computational models in machine learning and cognitive science that mimic the structure and function of biological neural networks (the central nervous system of animals, especially the brain) to estimate or approximate functions. Artificial neural networks can include convolutional neural networks (CNNs), deep neural networks (DNNs), multilayer perceptrons (MLPs), and other types of neural networks.

[0030] The working principle of each layer in a neural network can be expressed mathematically. To describe, among which, The sample (i.e., input data) typically has multiple attributes, so it is usually a vector; w represents the weight value of a neuron in the neural network; b is the bias; and a is the activation function. It is the output vector. From a physical perspective, the work of each layer in a neural network can be understood as transforming the input space (the set of input vectors) to the output space (i.e., from the row space to the column space of a matrix) through five operations on the input space. These five operations include: 1. Dimensionality increase / decrease; 2. Magnification / scaling; 3. Rotation; 4. Translation; 5. "Bending". Among them, operations 1, 2, and 3 are... Operation 4 is completed by +b, and operation 5 is implemented by a. The term "space" is used here because the objects being classified are not individual things, but a class of things; space refers to the collection of all individuals within this class of things. The variable w determines the spatial transformation from the input space to the output space described above; that is, the w at each layer controls how the space is transformed.

[0031] Below is a brief introduction to the method of performing neural network calculations using XB.

[0032] The following shows a weight applicable to this application, which is a j-row k-column matrix, where each element represents a weight value.

[0033]

[0034] Weights can be configured in the XB before neural network computation. For example, each element in a matrix is ​​configured in a corresponding cell in the XB, so that multiplication and addition operations can be performed on the input data and the matrix representing the weights through the XB.

[0035] XB is typically a multiply-accumulate array constructed from non-volatile storage media such as resistive random-access memory (ReRAM). The structure of an XB is as follows: Figure 1 As shown. An XB can include multiple units, such as G. 1,1 G 2,1 The cell is located at the intersection of a row and a column. If an XB includes 1000 rows and 1000 columns, then the number of cells in the XB is one million. In this embodiment of the invention, the weight value W can be... 0,0 Configure to Figure 1 G 1,1 In the middle, the weight value W 1,0 Configure to Figure 1 G 2,1Medium, etc. Each weight value corresponds to a cell, and each weight value is stored in the cell in the form of conductance. During neural network computation, input data is transmitted through the word lines of XB (such as...). Figure 1 Input XB is input through input port 104 (as shown). Input data can be represented by voltages (such as V1, V2, V3, and V4), allowing the input data to be multiplied by the weight values ​​stored in the cell. The calculated result is then emitted as current from the output of each column of XB (such as...). Figure 1 The output port 106 shown is the output port.

[0036] The current value of each column output can be calculated using the formula I = GV. For example, the current value of the first column output is denoted as I1, and I1 = V1 * G. 1,1 +V2*G 2,1 +V3*G 3,1 +V4*G 4,1 .

[0037] Due to limitations in device manufacturing processes, the data range that an XB cell can represent is relatively small; that is, the cell precision of an XB is limited. For example, ReRAM typically supports 16 different conductance states and can represent 4-bit weight values, meaning that the maximum cell precision of a ReRAM-based XB is 4 bits. XB cells based on embedded flash (eFlash) typically have a cell precision of 6–8 bits. This limited cell precision necessitates that the weights of the neural network be quantized before being written into the cells.

[0038] In this application, quantization refers to a change that amplifies or reduces the input data or weight values ​​of a neural network to a specific precision range. Typically, amplification or reduction can be achieved by multiplying the input data or weight values ​​by a constant and then rounding. If the amplified or reduced value still exceeds the precision range, the maximum or minimum value of that range is used to represent the input data or weight values.

[0039] For example, the precision range of 8 bits is [-128, 127]. The input data 4.64 can be multiplied by 16 and then rounded to obtain the quantized value 74. The constant multiplied during quantization, whether amplified or reduced, is called the quantization factor, such as 16 in this example.

[0040] Figure 2 This is an example of the impact of unit precision on neural network computation.

[0041] W, W 4bit:4x and W 8bit:64x There are three matrices, corresponding to three weights, where W is the original weight; W 4bit:4x This is the first type of quantization weight, with a quantization coefficient of 4, corresponding to a cell precision of 4 bits; W 8bit:64xThis is the second type of quantization weight, with a quantization coefficient of 64 and a corresponding unit precision of 8 bits.

[0042] For the two elements 0.060582 and 0.112672 in W, 4-bit quantization yields a result of 0, which is also the actual value represented by 0. 8-bit quantization yields 4 and 7, representing actual values ​​of 0.0625 (4 / 64) and 0.109375 (7 / 64), respectively. This shows that the 8-bit quantization result (i.e., the high-bit quantization result) is closer to the original weight value than the 4-bit quantization result (i.e., the low-bit quantization result). In other words, the higher the unit precision (i.e., the number of quantization bits), the lower the quantization error, and the higher the accuracy of the corresponding neural network calculation result. However, due to limitations in device manufacturing processes, the negative impact of quantization error on the neural network calculation result cannot be reduced by indefinitely increasing the unit precision.

[0043] Besides the impact of unit precision on neural network computation, TIA also affects neural network computation. TIA is a component of the analog-to-digital converter (ADC), used to convert the analog signal output from XB (such as...) into digital signal representations. Figure 1 The I1 in the signal is converted into a digital signal.

[0044] Figure 3 This is an example of the impact of TIA accuracy on neural network computation.

[0045] X 8bit:16x This represents input data with a quantization factor of 16, provided the cell precision is 8 bits. 8bit:64x This indicates a quantization weight with a quantization factor of 64, provided the unit precision is 8 bits. 26bit:1024x This represents the output result corresponding to the current obtained after the XB multiplication and addition operation. After input quantization and weighted quantization, the result obtained by XB is amplified by 1024 times. Therefore, 2369 actually represents 2.3135, and -7629 actually represents -7.4502.

[0046] If the TIA precision is 2 -a The digital signal obtained by the current passing through the ADC can be equivalent to Y 26bit:1024x Shift right by 'a' positions. Figure 3 Middle Y 8bit:2x and Y 8bit:0.125x These represent the results obtained by the ADC when a = 9 and a = 13, respectively, where Y 8bit:2x The results 4 and -15 actually represent 2 (4 / 2) and -7.5 (-15 / 2); Y 8bit:0.125xThe results 0 and -1 actually represent 0 and -8 (-1 / 0.125). Comparing the difference between 2.3135 and 2 and the difference between -7.4502 and -7.5, or comparing the difference between 2.3135 and 0 and the difference between -7.4502 and -8, we can draw the following conclusion: With the same TIA precision, Y... 26bit:1024x The smaller the absolute value of TIA, the greater the ADC error caused by TIA precision. Therefore, to reduce the ADC error caused by TIA precision, it is necessary to amplify the output of XB. This can be achieved by amplifying the weight values ​​(e.g., quantizing the original weight values ​​into high-bit weight values); however, amplifying the weight values ​​may cause the amplified weight values ​​to exceed the cell precision.

[0047] To address the aforementioned problems, this application provides a method for neural network computation that can reduce quantization error and ADC error without changing the unit precision.

[0048] Before performing the neural network computation method provided in this application, the initial weight values ​​of the neural network need to be preprocessed, including quantization and splitting.

[0049] The processor can quantize the initial weight values ​​in the initial weights of the neural network to obtain the quantized initial weights, and then... Figure 4 The method shown performs the split.

[0050] S410: Compare the cell precision with the quantized initial weight values. If any quantized initial weight value falls within the numerical representation range of the cell precision, then directly deploy the quantized initial weights on XB. If some or all of the quantized initial weight values ​​exceed the numerical representation range of the cell precision, then execute S420.

[0051] For example, the first initial weight value is a quantized initial weight value. If the number of bits in the first initial weight value is m, then the first initial weight value can be represented by W. m This means that if the unit precision of XB is n bits, then when |W m |≤2 n-1 When |W| is -1, the first initial weight value does not need to be split; when ... m |>2 n-1 When the initial weight is -1, the first initial weight value needs to be split.

[0052] S420 represents the initial weight values ​​after splitting and quantizing.

[0053] Taking the first initial weight value as an example, when 2 n-1 -1<|W m |≤p(2 n-1 When -1), W can be used. m Split into There are several weight values, where p is the number of units that can represent the first initial weight value, and p is greater than 1. This indicates rounding up to the nearest integer. Of the weight values, the first Each weight value is sign(2) n-1 -1), the last weight value is Where sign represents W m The symbolic function.

[0054] When |W m |>p(2 n-1 When W = -1, it means that the first weight value exceeds the maximum numerical range that p units can represent, and W can be... m It is split into p weight values, each weight value is sign(2). n-1 -1).

[0055] The initial weight value can be split to obtain N (p is an example of N) target weight values. All N target weight values ​​belong to the numerical representation range supported by an XB unit. The sum of the N target weight values ​​is less than or equal to the initial weight value, where N is an integer greater than 1.

[0056] S430: After the above splitting steps are completed, the number of units M occupied by the quantized initial weights after splitting can be counted. If M does not exceed the number of rows of XB, S440 can be executed; if M exceeds the number of rows of XB, it is necessary to return to S420 and re-split the quantized initial weights.

[0057] Optionally, during the re-splitting process, the maximum number of units occupied by each quantized initial weight value after splitting can be gradually reduced. For example, in the first re-splitting process, this maximum number can be set to p-1, and in the second re-splitting process, this maximum number is p-2. The splitting stops when the number of units occupied by the quantized initial weight after splitting is less than or equal to the number of rows of XB, and S440 is executed.

[0058] The above-described splitting and re-splitting processes are illustrative examples. Other splitting methods that simultaneously meet the following conditions are also within the protection scope of this application.

[0059] Condition 1: Ensure that the number of cells occupied by the splitting result of the quantized initial weights does not exceed the number of rows of XB;

[0060] Condition 2: Ensure that no single weight value in the initial weight splitting result after quantization exceeds the maximum numerical range represented by an XB unit.

[0061] S440 generates a replication relationship based on the splitting method of each weight value in the quantized initial weights.

[0062] For example, if the quantized initial weights contain three initial weight values, where the first initial weight value w1 is not split, the second initial weight value w2 is split into three weight values, and the third initial weight value w3 is split into two weight values, then the replication relationship is in the form of the table below.

[0063] Table 1

[0064] w1 1 w2 3 w3 2

[0065] The replication relationship can also be in the form of a piecewise function, implemented by a piecewise function circuit. This application does not limit the specific form of the replication relationship.

[0066] S450, the initial weights after quantization are split to obtain the target weights. After the above steps, if the target weights meet the row count and accuracy requirements of XB, then the target weights can be deployed on XB.

[0067] The initial weights after quantization require more bits to represent, while the target weights can be represented using fewer bits. Therefore, the initial weights after quantization can be called high-bit weights, and the target weights can be called low-bit weights.

[0068] Before performing neural network computation, the original weights can be quantized into high-bit weights to reduce quantization and ADC errors. Then, the splitting method provided in this application is used to split the high-bit weights into low-bit weights that meet the cell precision requirements. These low-bit weights (target weights) are then deployed on the XB. Therefore, the preprocessing method provided in this application can reduce quantization and ADC errors without changing the cell precision.

[0069] Once the target weights are deployed, execution can proceed. Figure 5 The neural network computation method shown is described. This method can be performed by a neural network circuit containing a computation array. The method includes the following:

[0070] S510 receives the first input data.

[0071] The first input data can be a single data point or multiple data points.

[0072] S520, the first input data is converted into second input data according to the replication relationship, wherein the second input data includes the first input data and at least a portion of the data in the first input data, and the replication relationship is used to determine the number of copies of the at least a portion of the data.

[0073] For example, the first input data are x1, x2 and x3, where x1 corresponds to the initial weight value w1, x2 corresponds to the initial weight value w2, and x3 corresponds to the initial weight value w3. According to Table 1, the second input data obtained after the transformation of the first input data are x1, x2, x2, x2, x3 and x3. That is, x1 is not copied, x2 is copied twice, and x3 is copied once.

[0074] After obtaining the second input data, the computation array can perform the following steps.

[0075] S530, perform neural network calculations on the second input data based on the target weights of the neural network set in the computing array.

[0076] Due to limitations in device manufacturing processes, the data range that a cell in a computing array can represent is finite; that is, the cell precision of the computing array is limited. To deploy larger weights on a computing array with relatively small cell precision, the original weights need to be split to obtain the target weights and replication relationships. Subsequently, the neural network computation method provided in this application is applied to determine the number of copies of the first input data based on the replication relationships. The first input data is then converted into second input data and used for neural network computation with the target weights. This enables neural network computation of large weights on a computing array with relatively small cell precision, improving the performance of the neural network.

[0077] Figure 6 The effect of two different weights on the computation results of the neural network is shown.

[0078] In current neural network circuits, the unit precision of XB is 8 bits, and the TIA precision of the ADC is 2. -9 The precision of a digital-to-analog converter (DAC) is 8 bits.

[0079] Without using the preprocessing method provided in this application, the floating-point inputs 0.278 and 0.402 are quantized to 4 and 6 respectively, with a quantization factor of 16, meaning the floating-point inputs are amplified by a factor of 16. The floating-point weights (i.e., the original weights) [1.025, -6.03, 2.278, -1.175] are quantized to [16, -96, 36, -28] respectively, with a quantization factor of 16, meaning the floating-point weights are amplified by a factor of 16. After multiplication and addition operations, the quantized input data and weights yield intermediate results 280 and -552. These intermediate results are then processed by the ADC and shifted right by 9 bits to obtain 0 and -2. Since 0 and -2 are the output results corresponding to the quantized and amplified inputs and weights, they still need to undergo quantization restoration processing. The quantization restoration factor is 2, and the actual output results are 0 and -4.

[0080] Using the preprocessing method provided in this application, the floating-point inputs 0.278 and 0.402 are quantized to 4 and 6 respectively, with a quantization factor of 16, meaning the floating-point inputs are amplified by a factor of 16; the floating-point weights are quantized to [66, -386, 146, -114], with a quantization factor of 64, meaning the floating-point inputs are amplified by a factor of 64. Subsequently, the weight value 146 is split into 127 and 19, and the weight value -368 is split into three -127 values, as follows. Figure 6 As shown in bold in the text. During computation, the quantized input data [4, 6] is expanded to [4, 4, 4, 6, 6], where 4 is copied twice and 6 is copied once, as shown in bold in the text. Figure 6 As shown in bold in the image, the product of the quantized input data and the split weights is 1140 and -2208. After processing by the ADC, 1140 and -2208 are shifted 9 bits to the right, resulting in 2 and -5. Since 2 and -5 are the output results corresponding to the input and weights after quantization amplification, they still need to undergo quantization restoration. The quantization restoration factor is 0.5, at which point 2 and -5 actually represent the values ​​1 and -2.5, respectively.

[0081] The floating-point output results are 1.20 and -2.38. Obviously, 1 and -2.5 are closer to the floating-point output results. Therefore, the calculation results obtained by using the preprocessing method and neural network calculation method provided in this application are closer to the floating-point output results. In other words, the error caused by the preprocessing method and neural network calculation method provided in this application is smaller.

[0082] The foregoing has detailed examples of the neural network computation method and preprocessing method provided in this application. It is understood that the corresponding apparatus, in order to achieve the above functions, includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0083] This application can divide the neural network computing device and preprocessing device into functional units based on the above method examples. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0084] Figure 7 This is a schematic diagram of the structure of a neural network processing device 700 provided in this application. The neural network processing device 700 includes a neural network computing device 710 and a preprocessing device 720. The neural network computing device 710 includes a receiving unit 711, a copying unit 712, a computing unit 713, and an output unit 714. The preprocessing device 720 includes a receiving unit 721, a quantization unit 722, and a splitting unit 723.

[0085] The receiving unit 721 is used to: receive the initial weights;

[0086] Quantization unit 722 is used to: quantize the initial weights to obtain quantized initial weights;

[0087] The splitting unit 723 is used to: split the quantized initial weights to obtain the target weights and replication relationships.

[0088] Optionally, the receiving unit 721 can directly receive the quantized initial weights and send them to the splitting unit 723, thus eliminating the need for further processing by the quantization unit 722.

[0089] The receiving unit 711 is used to: receive first input data;

[0090] The copying unit 712 is configured to: convert the first input data into second input data according to a copying relationship, wherein the second input data includes the first input data and copy data of at least a portion of the first input data, and the copying relationship is used to determine the number of copy data of the at least a portion of the data;

[0091] The computing unit 713 is used to: perform neural network calculations on the second input data based on the target weights of the neural network set in the computing array;

[0092] The output unit 714 is used to output the results of the above neural network calculations.

[0093] The specific working methods and beneficial effects of the neural network computing device 710 and the preprocessing device 720 can be found in the relevant descriptions in the method embodiments, and will not be repeated here.

[0094] Optionally, the receiving unit 711, receiving unit 721, and output unit 714 can be integrated together, with their functions implemented by the communication unit; the copying unit 712, quantization unit 722, and splitting unit 723 can be integrated together, with their functions implemented by the processor. Based on this form, this application also provides another neural network processing device, such as... Figure 8 As shown.

[0095] Figure 8 The dashed lines in the diagram indicate that the unit or module is optional. The device 800 can be used to implement the methods described in the above method embodiments, and the device 800 can be a terminal device, a server, or a chip.

[0096] The apparatus 800 includes one or more processors 801 that can support the implementation of the methods in the method embodiments. The processor 801 can be a general-purpose processor or a special-purpose processor. For example, the processor 801 can be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gate, transistor logic devices, or discrete hardware components.

[0097] The processor 801 can be used to control the device 800, execute software programs, and process data from the software programs. The device 800 may also include a communication unit 805 for inputting (receiving) and outputting (transmitting) signals.

[0098] For example, device 800 may be a chip, communication unit 805 may be the input and / or output circuit of the chip, or communication unit 805 may be the communication interface of the chip, and the chip may be a component of terminal equipment, network equipment or other electronic equipment.

[0099] For example, device 800 may be a terminal device or a server, and communication unit 805 may be a transceiver of the terminal device or the server, or communication unit 805 may be a transceiver circuit of the terminal device or the server.

[0100] The device 800 may include one or more memories 802 storing a program 804. The program 804 can be executed by a processor 801 to generate instructions 803, causing the processor 801 to execute the method described in the above method embodiments according to the instructions 803. Optionally, the memory 802 may also store data (such as the various weights and input data described above). Optionally, the processor 801 may also read data stored in the memory 802, which may be stored at the same memory address as the program 804, or the data may be stored at a different memory address than the program 804.

[0101] The processor 801 and memory 802 can be configured separately or integrated together, for example, integrated on the system-on-chip (SOC) of the terminal device.

[0102] The device 800 may also include a computing array 806, such as the XB described above. Data output from the processor 801 is processed by the DAC 807 and then loaded onto the computing array 806 as input data. Subsequently, the computing array 806 performs neural network calculations on the input data and pre-configured target weights to obtain preliminary calculation results. These preliminary calculation results are then processed by the ADC 808 and output as the final calculation result.

[0103] This application also provides a computer program product that, when executed by processor 801, implements the methods described in any of the method embodiments of this application.

[0104] The computer program product can be stored in memory 802, for example, program 804. Program 804 is finally converted into an executable object file that can be executed by processor 801 after processing such as preprocessing, compilation, assembly and linking.

[0105] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer, implements the methods described in any of the method embodiments of this application. The computer program may be a high-level language program or an executable object program.

[0106] The computer-readable storage medium is, for example, memory 802. Memory 802 can be volatile memory or non-volatile memory, or memory 802 can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0107] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process and technical effects of the above-described apparatus and equipment can be referred to the corresponding processes and technical effects in the foregoing method embodiments, and will not be repeated here.

[0108] In the several embodiments provided in this application, the systems, apparatuses, and methods disclosed can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not performed. The apparatus embodiments described above are merely illustrative; the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components can be combined or integrated into another system. Furthermore, the coupling between units or components can be direct coupling or indirect coupling, including electrical, mechanical, or other forms of connection.

[0109] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0110] Furthermore, the terms "system" and "network" are often used interchangeably in this paper. The term "and / or" in this paper merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this paper generally indicates that the preceding and following related objects have an "or" relationship.

[0111] In summary, the above description is merely a preferred embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A neural network computation method, characterized in that, Applied to cross arrays XB, including: Receive the first input data; The first input data is converted into second input data according to the replication relationship, wherein the second input data includes the first input data and at least a portion of the first input data's replicated data. The replication relationship is used to determine the number of replicated data of the at least a portion of the data. The replication relationship is determined based on the data representation range to which the quantized initial weight value corresponding to the at least a portion of the data belongs and the data representation range supported by the computing array. The target weights of the neural network set in the computation array of the cross array XB and the second input data are multiplied and added. The multiplication and addition operation is a dot product operation between each weight value in the target weights and the second input data. Each weight value corresponds to a unit of the XB.

2. The method according to claim 1, characterized in that, Also includes: The initial weight values ​​in the initial weights of the neural network are quantized to obtain quantized initial weights, wherein at least a portion of the quantized initial weight values ​​do not belong to the data representation range supported by the computing array. The target weight is determined based on the quantized initial weight, wherein the target weight includes the result of splitting at least a portion of the initial weight values ​​in the quantized initial weight, and any target weight value in the target weight belongs to the data representation range supported by the computing array.

3. The method according to claim 2, characterized in that, The quantized initial weight value includes a first initial weight value, and determining the target weight based on the quantized initial weight includes: The first initial weight value is split into N target weight values, all of which belong to the data representation range supported by the computing array. The sum of the N target weight values ​​is less than or equal to the first initial weight value, and N is an integer greater than 1.

4. The method according to any one of claims 1 to 3, characterized in that: The number of rows for the target weight is less than or equal to the total number of rows in the computation array.

5. A neural network computing device, characterized in that, Applied to cross arrays XB, including: A receiving unit is used to receive the first input data; A copying unit is used to convert the first input data into second input data according to a copying relationship, wherein the second input data includes the first input data and copy data of at least a portion of the first input data, and the copying relationship is used to determine the number of copy data of the at least a portion of the data, and the copying relationship is determined according to the data representation range to which the quantized initial weight value corresponding to the at least a portion of the data belongs and the data representation range supported by the computing array; The calculation unit is used to perform a multiplication-addition operation on the target weights of the neural network set in the calculation array of the cross array XB and the second input data. The multiplication-addition operation is a dot product operation between each weight value in the target weights and the second input data, and each weight value corresponds to a unit of the XB.

6. The apparatus according to claim 5, characterized in that, Also includes: A quantization unit is used to quantize the initial weight values ​​in the initial weights of the neural network to obtain quantized initial weights, wherein at least a portion of the quantized initial weight values ​​do not belong to the data representation range supported by the computing array. A splitting unit is used to determine the target weight based on the quantized initial weights, wherein the target weights include the result of splitting at least a portion of the initial weight values ​​in the quantized initial weights, and any one of the target weight values ​​belongs to the data representation range supported by the computing array.

7. The apparatus according to claim 6, characterized in that, The quantized initial weight value includes a first initial weight value, and the splitting unit is specifically used for: The first initial weight value is split into N target weight values, all of which belong to the data representation range supported by the computing array. The sum of the N target weight values ​​is less than or equal to the first initial weight value, and N is an integer greater than 1.

8. The apparatus according to any one of claims 5 to 7, characterized in that: The number of rows for the target weight is less than or equal to the total number of rows in the computation array.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the method of any one of claims 1 to 4.

Images