Hardware Implementation Method, Device, Equipment and Medium of Neural Network Batch Normalization Layer

Abstract

The present application relates to the field of neural network computing technology, and in particular to a method, device, equipment and medium for hardware implementation of a neural network batch normalization layer, wherein the method comprises: in a neural network, determining the current convolution result of the neural network; generating a K matrix based on the current convolution result, and obtaining a memristor array that is mapped to the K matrix, wherein the conductance difference of the memristor array corresponds to the parameters of the K matrix; performing BN layer calculation of the neural network using the memristor array to obtain the calculation result of the BN layer. Thus, the problem that the BN layer calculation in the related art is only applicable to binary neural networks and is not suitable for high-precision neural network hardware implementation is solved. By implementing the BN layer calculation on the memristor array, the data transmission back and forth between the processor unit and the memristor array unit is saved, and the system energy efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of neural network computing technology, and particularly to a method, device, equipment and medium for hardware implementation of a neural network batch normalization layer. Background Technique

[0002] In the related technology, the hardware implementation of the BN layer (Batch Normalization layer) includes the following steps:

[0003] (1) In a binary neural network, the operation of judging the sign bit is combined with the BN layer calculation. The output result of the convolutional layer only needs to be compared with Y TH . If it is greater than this value, the final result is 1. If it is less than this value, the result is -1;

[0004] (2) Fuse the BN layer parameters into the previous convolutional layer:

[0005]

[0006] Z N = X N * W N '+ bias';

[0007] Take as the equivalent weight matrix, with the input and output unchanged, take N ' and bias' as the equivalent bias, map W N ,1] The converted voltage pulse signal is input to the corresponding bit line terminal. The current flowing through the source line is also the result after the convolution of this layer and the calculation of the BN layer;

[0008] (3) Through the lookup table method based on the memristor array. For the simple output of the binary neural network, for different convolution results, store the corresponding BN layer calculation results in the memristor array in the form of a lookup table.

[0009] (4) Implement the BN layer calculation through a 16-bit adder and multiplier unit.

[0010] However, the related technology is only applicable to binary neural networks and is not suitable for high-precision neural network hardware implementation. In addition, the hardware implementation of the BN layer is mainly implemented in the CPU (central processing unit) or GPU (graphics processing unit). With the gradual development of storage-computing integrated technology, if the BN layer is still placed in a general processing unit for calculation, the data transfer between the storage-computing integrated module and the CPU / GPU will hinder the further improvement of the system energy efficiency, which needs to be solved urgently.

[0011] Application Contents

[0012] The present application provides a method, apparatus, device and medium for hardware implementation of a neural network batch normalization layer to solve the problem that BN layer calculation is only applicable to binary neural networks and is not suitable for hardware implementation of neural networks with higher precision in the related art. By implementing BN layer calculation on a memristor array, the back-and-forth transmission of data between a processor unit and a memristor array unit is saved, thereby improving the energy efficiency of the system.

[0013] The first aspect of the present application provides a method for hardware implementation of a neural network batch normalization layer, comprising the following steps:

[0014] In a neural network, determining a current convolution result of the neural network;

[0015] Generate a K matrix based on the current convolution result, and obtain a memristor array that is mapped to the K matrix, wherein the conductance difference of the memristor array corresponds to a parameter of the K matrix; and

[0016] The memristor array is used to perform BN layer calculation of the neural network to obtain calculation results of the BN layer.

[0017] Optionally, obtaining a memristor array in a mapping relationship with the K matrix includes:

[0018] Any two memristor units in the memristor array are combined into a 2T2R unit to obtain the memristor array.

[0019] Optionally, source lines of any two memristor units are connected, and voltage pulse signals with the same amplitude and opposite polarities are applied to the bit lines.

[0020] Optionally, it also includes:

[0021] Detecting the actual current flowing through the source line;

[0022] The actual voltage and at least one conductance difference between the upper and lower conductances are calculated according to the actual current.

[0023] Optionally, performing the BN layer calculation of the neural network by using the memristor array includes:

[0024] Converting the parameters in the K matrix into the at least one conductance difference.

[0025] An embodiment of the second aspect of the present application provides a hardware implementation device for a neural network batch normalization layer, including:

[0026] A determination module, configured to determine a current convolution result of the neural network in the neural network;

[0027] An acquisition module, configured to generate a K matrix based on the current convolution result, and obtain a memristor array having a mapping relationship with the K matrix, where a conductance difference of the memristor array corresponds to a parameter of the K matrix; and

[0028] A first calculation module, configured to perform the BN layer calculation of the neural network by using the memristor array to obtain a calculation result of the BN layer.

[0029] Optionally, the acquisition module is specifically configured to:

[0030] Forming a 2T2R unit by any two memristor cells in the memristor array to obtain the memristor array.

[0031] Optionally, source lines of any two memristor cells are connected, and voltage pulse signals with the same amplitude and opposite polarities are applied to the bit lines.

[0032] Optionally, it further includes:

[0033] A detection module, configured to detect an actual current flowing through the source line;

[0034] A second calculation module, configured to calculate at least one of an actual voltage and a conductance difference of upper and lower conductances according to the actual current.

[0035] Optionally, the first calculation module is specifically configured to:

[0036] Converting the parameters in the K matrix into the at least one conductance difference.

[0037] An embodiment of the third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the hardware implementation method of the neural network batch normalization layer as described in the above embodiment.

[0038] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored, and the program is executed by a processor to implement the hardware implementation method of the neural network batch normalization layer.

[0039] Therefore, the current convolution result of the neural network can be determined in the neural network, and a K matrix can be generated based on the current convolution result, and a memristor array with a mapping relationship with the K matrix can be obtained, and the memristor array can be used to perform BN layer calculations of the neural network to obtain the calculation results of the BN layer. Therefore, the BN layer calculation is converted into a matrix-vector multiplication operation, and its parameters can be deployed to the memristor array for storage and calculation, just like the forward connection and convolution layer, which solves the problem that the BN layer calculation in the related technology is only applicable to binary neural networks and is not suitable for high-precision neural network hardware implementation, and saves the data transmission back and forth between the processor unit and the memristor array unit, and improves the system energy efficiency.

[0040] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0042] Figure 1 A flowchart of a method for hardware implementation of a neural network batch normalization layer provided according to an embodiment of the present application;

[0043] Figure 2 This is an example diagram of a convolutional layer with a BN layer;

[0044] Figure 3 is an example diagram of a convolution operation and a convolution operation based on a memristor array according to an embodiment of the present application;

[0045] Figure 4 This is an example diagram of performing BN layer calculation based on a memristor array according to an embodiment of the present application;

[0046] Figure 5 An exemplary diagram of a hardware implementation device for a neural network batch normalization layer according to an embodiment of the present application;

[0047] Figure 6 is an exemplary diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0049] The following describes the hardware implementation method, device, equipment and medium of the neural network batch normalization layer of the embodiment of the present application with reference to the accompanying drawings. In view of the problems that the BN layer calculation in the related technology mentioned in the background technology center is only applicable to binary neural networks and is not suitable for high-precision neural network hardware implementation, the present application provides a neural network batch normalization layer hardware implementation method, in which the current convolution result of the neural network can be determined in the neural network, and a K matrix is generated based on the current convolution result, and a memristor array with a mapping relationship with the K matrix is obtained, and the memristor array is used to perform the BN layer calculation of the neural network to obtain the calculation result of the BN layer. Thus, the BN layer calculation is converted into a matrix-vector multiplication operation, and like the forward connection and convolution layer, its parameters can be deployed to the memristor array for storage and calculation, solving the problems that the BN layer calculation in the related technology is only applicable to binary neural networks and is not suitable for high-precision neural network hardware implementation, saving the data transmission back and forth between the processor unit and the memristor array unit, and improving the system energy efficiency.

[0050] Specifically, Figure 1 A flowchart of a method for hardware implementation of a neural network batch normalization layer provided in an embodiment of the present application.

[0051] like Figure 1 As shown, the hardware implementation method of the neural network batch normalization layer includes the following steps:

[0052] In step S101, in the neural network, the current convolution result of the neural network is determined.

[0053] It should be understood that in a neural network, as the number of layers increases, the output values of the hidden layer neurons will change, and the overall distribution may shift toward the boundary of the activation function value range, causing the low-level network gradient to disappear during training and the overall convergence speed to slow down. The BN layer is a common processing module in neural networks and is generally placed before the activation function ReLU. It can unify the discrete output to the interval where the activation function is more sensitive to the input, avoid the gradient disappearance, and reduce the difficulty of network training. Figure 2 As shown, Figure 2 Schematic diagram of the convolutional layer of the BN layer.

[0054] For ease of understanding, the embodiment of the present application takes the implementation of the convolutional layer with BN calculation as an example. The parameters of the BN layer include β, γ, σ, and μ. Among them, β and γ are trainable parameters that gradually converge during the training process, while the two parameters σ and μ represent the standard deviation and mean of the output value, respectively, and are related to the training samples. After the training is completed, in the test phase, the relevant parameters of the BN layer remain unchanged.

[0055] Assume X N is the input vector of the Nth layer, W Nis the weight matrix obtained by unfolding the N convolutional kernels of the Nth convolutional layer, as Figure 3 (b) shows, Figure 3 (b) is an example diagram of a convolutional operation based on a memristor array, Figure 3 (a) is an example diagram of a convolutional operation.

[0056] Y N = W N * X N ;

[0057] Assume that the output vector Y of this layer N The vector after being processed by the BN layer is Z N , and the calculation formula is:

[0058]

[0059]

[0060]

[0061] where ε is a negligible minimum value introduced to avoid division by zero.

[0062] Furthermore, assume that the Nth convolutional layer has n convolutional kernels, corresponding to n output neurons y1, y2, y n .

[0063]

[0064] In the existing in-memory computing scheme based on a memristor array, n convolutional kernels are unfolded into a row and mapped to n rows of the memristor array. The BN layer parameters corresponding to each output neuron are k1, b1, k2, b2, ……, kn, bn. The BN layer calculation formula is as follows:

[0065]

[0066]

[0067] In step S102, generate a K matrix based on the current convolution result, and obtain a memristor array that has a mapping relationship with the K matrix, where the conductance difference of the memristor array corresponds to the parameters of the K matrix.

[0068] Optionally, in some embodiments, obtaining a memristor array that has a mapping relationship with the K matrix includes: forming 2T2R cells by any two memristor units in the memristor array to obtain the memristor array.

[0069] Optionally, in some embodiments, the source lines of any two memristor units are connected, and voltage pulse signals with the same amplitude and opposite polarities are applied on the bit lines.

[0070] Specifically, the embodiment of the present application can convert the parameters obtained in step S101 (i.e., the current convolution result) into matrices K and The product of vectors.

[0071]

[0072]

[0073] Furthermore, the embodiment of the present application can map the K matrix to the memristor array. For example, two memristor units are used to form a 2T2R unit. Since the source lines of the two units are connected, the voltage pulse signals with the same amplitude and opposite polarity are applied to the bit line.

[0074] Optionally, in some embodiments, the hardware implementation method of the neural network batch normalization layer further includes: detecting an actual current flowing through the source line; and calculating an actual voltage and at least one conductance difference between upper and lower conductances based on the actual current.

[0075] That is to say, according to Kirchhoff's current law and Ohm's law, the current value flowing through the source line in the embodiment of the present application is the voltage value and the difference between the upper and lower conductances.

[0076] In step S103, the memristor array is used to perform BN layer calculation of the neural network to obtain the calculation result of the BN layer.

[0077] Optionally, in some embodiments, using a memristor array to perform BN layer calculations of a neural network includes: converting parameters in a K matrix into at least one conductance difference value.

[0078] Specifically, the embodiment of the present application converts the parameters in the K matrix into two conductance difference values. If the 0 value in the matrix is to be mapped to the array, both conductances can be adjusted to a high impedance state.

[0079] Thus, the BN layer calculation is converted into the product of the matrix and the vector, that is, the BN layer calculation is implemented on the memristor array, which saves the data transmission back and forth between the processor unit and the memristor array unit, and improves the system energy efficiency. It should be noted that in addition to the above-mentioned method of using 2T2R to obtain the memristor array, the embodiment of the present application can also use other storage units such as SRAM, DRAM, and the method of implementing the BN layer calculation is consistent with the above, and in order to avoid redundancy, it will not be described in detail here.

[0080] In order to facilitate those skilled in the art to further understand the hardware implementation method of the neural network batch normalization layer in the embodiment of the present application, it is elaborated in detail below in combination with specific embodiments.

[0081] like Figure 4 As shown,Figure 4 Only the last column is the bias value b, while the diagonals of the first few rows are the k values corresponding to each neuron (solid circles), and all other elements in the matrix are 0 (hollow circles).

[0082] During the test, the conductance in the array does not need to be changed. The results of the Nth layer of convolutional network are input into the bit lines of the first few rows (for example, 8 bits of data are converted into 8 voltage pulse signals, corresponding to the value of each bit). The last column maps the bias b, so the number of pulse signals corresponding to the bit line input 1. For example, the maximum value of the convolution output is y max , mapping it to 255 pulse signals. Apply round(y max / 255) voltage pulse signals with the same amplitude and width (round is the rounding function). The source line of the memristor is connected to the ADC, and the quantized value is the calculation result of the BN layer.

[0083] According to the hardware implementation method of the neural network batch normalization layer proposed in the embodiment of the present application, the current convolution result of the neural network can be determined in the neural network, and a K matrix can be generated based on the current convolution result, and a memristor array with a mapping relationship with the K matrix can be obtained, and the memristor array is used to perform the BN layer calculation of the neural network to obtain the calculation result of the BN layer. Thus, the BN layer calculation is converted into a matrix-vector multiplication operation, and its parameters can be deployed to the memristor array for storage and calculation, just like the forward connection and convolution layer, which solves the problem that the BN layer calculation in the related technology is only applicable to binary neural networks and is not suitable for high-precision neural network hardware implementation, etc., which saves the data transmission back and forth between the processor unit and the memristor array unit, and improves the system energy efficiency.

[0084] Next, a hardware implementation device for a neural network batch normalization layer proposed in an embodiment of the present application will be described with reference to the accompanying drawings.

[0085] Figure 5 It is a block diagram of a hardware implementation device of a neural network batch normalization layer according to an embodiment of the present application.

[0086] like Figure 5 As shown, the neural network batch normalization layer hardware implementation device 10 includes: a determination module 100, an acquisition module 200 and a first calculation module 300.

[0087] Wherein, the determination module 100 is used to determine the current convolution result of the neural network in the neural network;

[0088] The acquisition module 200 is used to generate a K matrix based on the current convolution result, and obtain a memristor array that is mapped to the K matrix, wherein the conductance difference of the memristor array corresponds to the parameters of the K matrix; and

[0089] The first calculation module 300 is used to perform BN layer calculation of the neural network using the memristor array to obtain the calculation result of the BN layer.

[0090] Optionally, the acquisition module 200 is specifically used for:

[0091] Any two memristor units in the memristor array are combined into a 2T2R unit to obtain a memristor array.

[0092] Optionally, source lines of any two memristor units are connected, and voltage pulse signals with the same amplitude and opposite polarities are applied to the bit lines.

[0093] Optionally, it also includes:

[0094] A detection module, used for detecting the actual current flowing through the source line;

[0095] The second calculation module is used to calculate the actual voltage and at least one conductance difference between the upper and lower conductances according to the actual current.

[0096] Optionally, the first calculation module 300 is specifically used for:

[0097] Convert the parameters in the K matrix to at least one conductance difference value.

[0098] It should be noted that the aforementioned explanation of the embodiment of the method for hardware implementation of the neural network batch normalization layer is also applicable to the hardware implementation device of the neural network batch normalization layer of this embodiment, and will not be repeated here.

[0099] According to the hardware implementation device of the neural network batch normalization layer proposed in the embodiment of the present application, the current convolution result of the neural network can be determined in the neural network, and a K matrix can be generated based on the current convolution result, and a memristor array with a mapping relationship with the K matrix can be obtained, and the memristor array can be used to perform BN layer calculations of the neural network to obtain the calculation results of the BN layer. Thus, the BN layer calculation is converted into a matrix-vector multiplication operation, and its parameters can be deployed to the memristor array for storage and calculation, just like the forward connection and convolution layer, which solves the problem that the BN layer calculation in the related technology is only applicable to binary neural networks and is not suitable for high-precision neural network hardware implementation, and saves the data transmission back and forth between the processor unit and the memristor array unit, and improves the system energy efficiency. .

[0100] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0101] A memory 601 , a processor 602 , and a computer program stored in the memory 601 and executable on the processor 602 .

[0102] When the processor 602 executes the program, it implements the hardware implementation method of the neural network batch normalization layer provided in the above embodiments.

[0103] Furthermore, the electronic device further includes:

[0104] A communication interface 603, which is used for communication between the memory 601 and the processor 602.

[0105] A memory 601, which is used to store computer programs that can run on the processor 602.

[0106] The memory 601 may include a high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0107] If the memory 601, the processor 602, and the communication interface 603 are implemented independently, the communication interface 603, the memory 601, and the processor 602 can be interconnected through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 6 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0108] Optionally, in a specific implementation, if the memory 601, the processor 602, and the communication interface 603 are integrated on a chip, the memory 601, the processor 602, and the communication interface 603 can communicate with each other through an internal interface.

[0109] The processor 602 may be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0110] This embodiment also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the above-mentioned hardware implementation method of the neural network batch normalization layer.

[0111] In the description of this specification, the description referring to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0112] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of this application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0113] Any process or method description shown in a flowchart or described in other ways herein can be understood as representing a module, segment, or part of code including one or more N executable instructions for implementing a customized logic function or process, and the scope of the preferred embodiments of this application includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in the reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of this application pertain.

[0114] The logic and / or steps represented in the flowchart or otherwise described herein can, for example, be considered a definitional sequence list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection part (electronic device) having one or N wirings, a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, as the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpretation, or otherwise processing as appropriate, and then storing it in a computer memory.

[0115] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), and the like.

[0116] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the method of implementing the above embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0117] In addition, each functional unit in various embodiments of the present application may be integrated into a processing module, may exist separately as individual physical units, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0118] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A hardware implementation method for a neural network batch normalization layer, characterized in that Including the following steps: In a neural network, determine the current convolution result of the neural network; Generate a K matrix based on the current convolution result, and obtain a memristor array in a mapping relationship with the K matrix, wherein the conductance difference of the memristor array corresponds to the parameters of the K matrix; and Use the memristor array to perform the calculation of the BN layer of the neural network to obtain the calculation result of the BN layer; Wherein, obtaining the memristor array in a mapping relationship with the K matrix includes: forming 2T2R units with any two memristor units in the memristor array to obtain the memristor array; The source lines of any two memristor units are connected, and voltage pulse signals with the same amplitude and opposite polarities are applied to the bit lines.

2. The method according to claim 1, wherein It further includes: Detect the actual current flowing through the source line; Calculate at least one of the actual voltage and the conductance difference between the upper and lower conductances according to the actual current.

3. The method according to claim 2, wherein The using the memristor array to perform the calculation of the BN layer of the neural network includes: Convert the parameters in the K matrix into the at least one conductance difference.

4. A hardware implementation device for a neural network batch normalization layer, characterized in that, It includes: A determination module, configured to determine the current convolution result of the neural network in the neural network; An acquisition module, configured to generate a K matrix based on the current convolution result, and obtain a memristor array in a mapping relationship with the K matrix, wherein the conductance difference of the memristor array corresponds to the parameters of the K matrix; And A first calculation module, configured to use the memristor array to perform the calculation of the BN layer of the neural network to obtain the calculation result of the BN layer; The acquisition module is specifically configured to: form 2T2R units with any two memristor units in the memristor array to obtain the memristor array; The source lines of any two memristor units are connected, and voltage pulse signals with the same amplitude and opposite polarities are applied to the bit lines.

5. The device according to claim 4, characterized in that, It further includes: A detection module, configured to detect the actual current flowing through the source line; A second calculation module, configured to calculate at least one of the actual voltage and the conductance difference between the upper and lower conductances according to the actual current.

6. The device according to claim 5, characterized in that, The first calculation module is specifically configured to: Convert the parameters in the K matrix into the at least one conductance difference.

7. An electronic device, characterized in that, It includes: A memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to implement the hardware implementation method of the neural network batch normalization layer as described in any one of claims 1-3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to be used to implement the hardware implementation method of the neural network batch normalization layer as described in any one of claims 1-3.

Images