Stacked resistive random access memory element layer optimization system and method using artificial intelligence technology

Abstract

The present disclosure provides a system and method for optimizing layers of stacked resistive random access memory elements using artificial intelligence technology. The method optimizes layers of stacked resistive random access memory elements using a neural network device with artificial intelligence technology, which can include the following steps: the neural network device classifies BNN parameters in a BNN model into physical parameters and hyperparameters; the neural network device obtains optimal parameters using the physical parameters and the hyperparameters; and the neural network device calculates the minimum channel size in the BNN model using the optimal parameters.

Description

Technical Field

[0001] This disclosure relates to a layer optimization system and method for stacked resistive random access memory (3DRRAM) devices utilizing artificial intelligence technology. More specifically, it relates to a layer optimization system and method for stacked resistive random access memory (3DRRAM) devices utilizing artificial intelligence technology, which obtains optimal parameters by classifying BNN model parameters in a binary neural network (BNN) model into physical parameters and hyperparameters, calculates the minimum channel value with high accuracy and minimal bias using the obtained parameters, and optimizes the layers of the stacked resistive random access memory (3DRRAM) using the calculated channel value. Background Technology

[0002] There is increasing interest in neuromorphic processors that mimic the human nervous system. Research has been conducted on implementing neuromorphic processors by designing neuronal and synaptic circuits that correspond to neurons and synapses found in the human nervous system. Such neuromorphic processors can be used to drive various neural networks, such as binary neural networks (BNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), and feedforward neural networks (FNNs), and can be applied to fields such as data classification or image recognition.

[0003] In recent years, with the development of neural network technology, research on using neural network devices in various electronic systems to analyze input data and extract effective information is actively underway.

[0004] Neural network devices require extensive computation on complex input data. To enable neural network devices to analyze input and extract data in real time, techniques capable of efficiently handling neural network computations are needed. In particular, given the limited resources of low-power, high-performance embedded systems such as smartphones, a technique is required that minimizes accuracy loss while reducing the computational load required to process complex input data. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] The technical problem to be solved by this disclosure is to provide a layer optimization system and method for stacked resistive random access memory (RRAM) devices utilizing artificial intelligence technology.

[0007] The technical problems to be solved by this disclosure are not limited to those described above. Other problems not mentioned can be clearly understood by those skilled in the art from the following description.

[0008] (II) Technical Solution

[0009] A layer optimization method for stacked resistive random access memory (SRAM) devices utilizing artificial intelligence technology, according to an embodiment of the present disclosure, the method uses a neural network device that optimizes the layers of the stacked SRAM device using artificial intelligence technology, and may include the following steps: the neural network device classifies BNN parameters in a BNN model into physical parameters and hyperparameters; the neural network device obtains optimal parameters using the physical parameters and the hyperparameters; and the neural network device calculates the minimum channel size in the BNN model using the optimal parameters.

[0010] The physical parameters may include the number of convolutional layers, channel (filter) size, kernel size, whether batch normalization is present, and whether pooling layers are present. The hyperparameters may include the optimizer, learning rate, and momentum.

[0011] The optimizer may include at least one of the following algorithms: Batch Gradient Descent, Stochastic Gradient Descent (SGD), Gradient Descent, Mini-Batch Gradient Descent, Momentum, Adagrad, Root Mean Square Propagation (RMSprop), and Adam.

[0012] The step of obtaining the optimal parameters may include the following steps: using the algorithm of the optimizer included in the hyperparameters to calculate the optimal algorithm; using the learning rate included in the hyperparameters to calculate the optimal learning rate for adjusting the weights and kernel update intensity in the BNN; and using the momentum included in the hyperparameters to calculate the optimal momentum by considering the momentum value in the BNN.

[0013] The step of calculating the optimal algorithm may include the following steps: calculating the optimal algorithm combination among at least one algorithm included in the optimizer; and calculating the ratio of the optimal algorithm combination.

[0014] The optimal algorithm can be such that, when the kernel size is 3×3, the ratio of Adam algorithm to SGD algorithm is 3:7, and when the kernel size is 5×5, the ratio of Adam algorithm to SGD algorithm is 6:4.

[0015] The same optimal learning rate can be achieved when the kernel size is 3×3 and when the kernel size is 5×5, and the optimal learning rate is 0.03.

[0016] The optimal momentum can be 0.5 when the kernel size is 3×3 and 0.6 when the kernel size is 5×5.

[0017] The step of obtaining the optimal parameters may include the following steps: determining whether the batch normalization and the pooling layer exist.

[0018] When the number of convolutional layers is 4, the channel size is 9, and the kernel size is 3×3, the batch normalization can be included in each convolutional layer.

[0019] When batch normalization is not included in each of the convolutional layers, the accuracy can differ by 8%.

[0020] The batch standardization can be performed using [Mathematical Formula 1].

[0021] [Mathematical Expression 1]

[0022]

[0023] When the number of convolutional layers is 4, the channel size is 9, and the kernel size is 3×3, the pooling layer can be located in the last two layers of the convolutional layers.

[0024] The step of calculating the minimum channel size may include the following steps: determining the accuracy of the minimum channel size.

[0025] With a minimum channel size of 8, the accuracy can be above 96% when the kernel size is 3×3.

[0026] The step of calculating the minimum channel size may further include the following steps: applying the error rate of each layer according to the minimum channel size to determine the accuracy of the minimum channel size.

[0027] When the minimum channel size is 8 and the kernel size is 3×3, the accuracy is 94.069% at the 40nm node, 93.777% at the 20nm node, and 93.07% at the 10nm node.

[0028] This disclosure may include the following steps: the neural network device optimizes the layers of the RRAM with minimum bias using the minimum channel size.

[0029] According to an embodiment of the present disclosure, a layer optimization system for stacked resistive random access memory (SRAM) devices utilizing artificial intelligence technology is provided. The system can execute a neural network method for optimizing the layers of stacked SRAM devices using artificial intelligence technology via a neural network device.

[0030] A program according to an embodiment of the present disclosure, combined with a computer as hardware and stored in a computer-readable recording medium, enables the execution of the layer optimization method for stacked resistive random access memory elements utilizing artificial intelligence technology.

[0031] Other specific matters in this disclosure are included in the detailed description and accompanying drawings.

[0032] (III) Beneficial Effects

[0033] According to this disclosure, artificial intelligence techniques can be used to optimize the layers of stacked resistive random access memory (RRAM) elements with minimal bias. Specifically, by minimizing the size of the 3D weight matrix, it is more effective in hardware processors using RRAM synapses.

[0034] According to this disclosure, it can be applied to robotic devices such as drones and advanced driver assistance systems (ADAS), as well as smart TVs, smartphones, medical devices, mobile devices, image display devices, measuring devices, Internet of Things (IoT) devices, etc. In addition, it can be mounted on at least one of various electronic devices, thereby enabling it to be applied to various electronic devices and provide high-quality services to a variety of users.

[0035] The effects of this disclosure are not limited to those mentioned above, and those skilled in the art can clearly understand other effects not mentioned from the following description. Attached Figure Description

[0036] Figure 1 This is a conceptual diagram illustrating a layer optimization system for stacked resistive random access memory elements utilizing artificial intelligence technology according to an embodiment of the present disclosure.

[0037] Figure 2This is a diagram illustrating a layer optimization method for stacked resistive random access memory elements utilizing artificial intelligence technology according to an embodiment of the present disclosure.

[0038] Figure 3 It is used for explanation Figure 2 The diagram shows the steps to obtain the optimal parameters.

[0039] Figure 4 and Figure 5 It is used for explanation Figure 3 The diagram shows the steps for calculating the optimal algorithm combination.

[0040] Figure 6 and Figure 7 It is used for explanation Figure 3 The diagram shows the steps for calculating the optimal learning rate.

[0041] Figure 8 and Figure 9 It is used for explanation Figure 3 The diagram shows the steps for calculating optimal momentum.

[0042] Figures 10 to 12 It is used for explanation Figure 3 The diagram shows the steps for determining whether a batch normalization and pooling layer exist.

[0043] Figure 13 and Figure 14 It is used for explanation Figure 2 The diagram shows the steps for calculating the minimum channel size.

[0044] Figure 15 It is used for explanation Figure 2 The diagram shows the steps for optimizing the RRAM layer with minimal bias.

[0045] Figure 16 It is possible. Figure 1 The diagram shows the hardware configuration of the computing device for the neural network apparatus. Detailed Implementation

[0046] The advantages and features of this disclosure, as well as the methods for implementing them, will become clear from reference to the accompanying drawings and the embodiments described in detail below. However, this disclosure is not limited to the embodiments disclosed below, but can be implemented in various different forms, and these embodiments are provided merely to complete this disclosure and to fully illustrate the scope of this disclosure to those skilled in the art to which this disclosure pertains, which is defined only by the scope of the claims.

[0047] The terminology used in this specification is for illustrative purposes only and is not intended to limit this disclosure.

[0048] In this specification, unless otherwise stated, the singular also includes the plural. The terms "comprises" and / or "comprising" as used in this specification do not exclude the presence or addition of one or more other components besides those described.

[0049] Throughout this specification, the same reference numerals denote the same components, and "and / or" includes all combinations of each and one or more of the described components.

[0050] Although terms such as "first" and "second" are used to describe various components, these components are not limited by these terms. These terms are used only to distinguish one component from another. Therefore, the first component described below can be the second component within the technical concept of this invention.

[0051] Unless otherwise defined, all terms used in this specification (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. Furthermore, unless explicitly defined otherwise, terms commonly used in dictionaries should not be interpreted ideally or excessively.

[0052] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0053] Figure 1 This is a conceptual diagram illustrating a layer optimization system for stacked resistive random access memory elements utilizing artificial intelligence technology according to an embodiment of the present disclosure.

[0054] like Figure 1 As shown, the layer optimization system 1 of the stacked resistive random access memory element utilizing artificial intelligence technology, as an embodiment of this disclosure, can optimize the parameters of the binary neural network (BNN) model through the neural network device 10 to minimize the filter size of the BNN model.

[0055] The following is for reference Figures 2 to 15 The operation of a layer optimization system 1 for stacked resistive random access memory elements utilizing artificial intelligence technology according to an embodiment of the present disclosure will be described.

[0056] Figure 2 This is a diagram illustrating a layer optimization method for stacked resistive random access memory elements utilizing artificial intelligence technology according to an embodiment of the present disclosure. Figure 3 It is used for explanation Figure 2 The diagram shows the steps to obtain the optimal parameters. Figure 4 and Figure 5 It is used for explanation Figure 3 The diagram shows a method for calculating the optimal algorithm combination. Figure 6 and Figure 7 It is used for explanation Figure 3 The figure shows the method for calculating the optimal learning rate. Figure 8 and Figure 9 It is used for explanation Figure 3 The figure shows the method for calculating optimal momentum. Figures 10 to 12 It is used for explanation Figure 3 The diagram shows the steps for determining whether a batch normalization and pooling layer exist. Figure 13 and Figure 14 It is used for explanation Figure 2 The diagram shows the method for calculating the minimum channel size. Figure 15 It is used for explanation Figure 2 The diagram illustrates a method for optimizing RRAM layers with minimal deviation. While this layer optimization method for stacked resistive random access memory elements utilizing artificial intelligence techniques is performed under coding conditions, it is not limited to this.

[0057] Additionally, referring to Table 1, implementation can be based on 100 iterations or 100 epochs under various conditions of kernel size, channel size, and convolutional layer, but is not limited to these.

[0058] Table 1

[0059]

[0060] First, such as Figure 2 As shown, the neural network device 10 can classify the BNN parameters in the BNN model into physical parameters and hyperparameters (S10).

[0061] The physical parameters may include, but are not limited to, the number of convolutional layers, the channel (filter) size, the kernel size, the presence of batch normalization, and the presence of pooling layers.

[0062] In addition, hyperparameters can include, but are not limited to, the optimizer, learning rate, and momentum. They can also include the number of hidden units, mini-batch size, number of hidden layers, and learning rate decay.

[0063] Next, the neural network device 10 can obtain the optimal parameters using the physical parameters and the hyperparameters (S20).

[0064] Specifically, such as Figure 3 As shown, the neural network device 10 can use the algorithm of the optimizer included in the hyperparameters to compute the optimal algorithm (S100).

[0065] The optimizer's algorithms can include: Batch Gradient Descent, which considers the entire dataset when calculating error; Stochastic Gradient Descent (SGD), which calculates only a randomly selected data point instead of the entire dataset when adjusting parameter values; Gradient Descent, which adjusts parameter values ​​by calculating only a specified amount; Mini-Batch Gradient Descent, which is faster than calculating the entire dataset and more stable than SGD; Momentum, which applies the laws of inertial physics; Adagrad, which applies different learning rates to each parameter; Root Mean Square Propagation (RMSProp), which improves upon the learning rate decline of Adagrad; and Adaptive Moment Estimation (Adam), which combines the advantages of RMSProp and momentum.

[0066] In this embodiment, to calculate the optimal algorithm, the neural network device 10 can utilize the SGD algorithm and the adaptive moment estimation algorithm to calculate the optimal algorithm combination. Although the SGD algorithm has a relatively low optimal accuracy, it improves speed by adjusting the weights of randomly extracted data instead of adjusting the weights of all data. The adaptive moment estimation algorithm combines the advantages of the root mean square propagation algorithm and the momentum algorithm. That is, the neural network device 10 can calculate the optimal algorithm combination where the ratio of the adaptive moment estimation algorithm to the SGD algorithm is 3:7 when the kernel size is 3×3, and 6:4 when the kernel size is 5×5.

[0067] In other words, the neural network device 10 can calculate the optimal ratio between the adaptive moment estimation algorithm and the SGD algorithm. The optimal ratio with the highest accuracy can be calculated by comparing the cases where the adaptive moment estimation algorithm is high and the SGD algorithm is high.

[0068] For example, the algorithm was repeatedly tested with ratios of 1:9 for the Adaptive Moment Estimation (IME) algorithm and SGD algorithm, 2:8 for IME, 3:7 for IME, 4:6 for IME, 5:5 for IME, 6:4 for IME, 7:3 for IME, 8:2 for IME, and 9:1 for SGD. The optimal algorithm combination with the optimal ratio was calculated, where the optimal ratio was 3:7 for a kernel size of 3×3 and 6:4 for a kernel size of 5×5.

[0069] Specifically, refer to Figure 4 It can be seen that, with a kernel size of 3×3, the accuracy of varying channel size when the number of convolutional layers is 2 (refer to...) Figure 4 (a) Accuracy of varying channel size when the number of convolutional layers is 4 (refer to) Figure 4 (b) and the accuracy of varying channel size when the number of convolutional layers is 6 (see [reference]). Figure 4 In (c), the highest accuracy is achieved when the ratio of adaptive moment estimation algorithm to SGD algorithm is 3:7.

[0070] Additionally, refer to Figure 5 It can be seen that, with a kernel size of 5×5, the accuracy of varying channel size when the number of convolutional layers is 2 (refer to...) Figure 5 (a) Accuracy of varying channel size when the number of convolutional layers is 4 (refer to) Figure 5 (b) and the accuracy of varying channel size when the number of convolutional layers is 6 (see [reference]). Figure 5 In (c), the highest accuracy is achieved when the ratio of adaptive moment estimation algorithm to SGD algorithm is 3:7.

[0071] Next, the neural network device 10 can use the learning rate included in the hyperparameters to calculate the optimal learning rate for adjusting the weights and kernel update intensity in the BNN (S110). That is, the neural network device 10 can calculate the same optimal learning rate of 0.03 for kernel sizes of 3×3 and 5×5.

[0072] For example, the neural network device 10 can compare and analyze the learning rate that represents how much learning is needed when learning from 0.01 to 0.1 once, and can calculate the optimal learning rate of 0.03 with an accuracy of 99%.

[0073] Specifically, refer to Figure 6 It can be seen that, with a kernel size of 3×3, the accuracy of varying channel size when the number of convolutional layers is 2 (refer to...) Figure 6 (a) Accuracy of varying channel size when the number of convolutional layers is 4 (refer to) Figure 6 (b) and the accuracy of varying channel size when the number of convolutional layers is 6 (see [reference]). Figure 6 In (c), the accuracy was highest when the learning rate was 0.03.

[0074] Additionally, refer to Figure 7 It can be seen that, with a kernel size of 5×5, the accuracy of varying channel size when the number of convolutional layers is 2 (refer to...) Figure 7 (a) Accuracy of varying channel size when the number of convolutional layers is 4 (refer to) Figure 7 (b) and the accuracy of varying channel size when the number of convolutional layers is 6 (see [reference]). Figure 7 In (c), the accuracy was highest when the learning rate was 0.03.

[0075] Next, the neural network device 10 can use the momentum included in the hyperparameters and consider the momentum value in the BNN to calculate the optimal momentum (S120). That is, the neural network device 10 can calculate the optimal momentum of 0.5 when the kernel size is 3×3 and 0.6 when the kernel size is 5×5.

[0076] For example, in order to reduce training time and calculate the optimal momentum value, the neural network device 10 can consider different momentum values ​​from 0.01 to 0.1 to calculate the optimal momentum of 0.5 when the kernel size is 3×3 and 0.6 when the kernel size is 5×5.

[0077] Specifically, refer to Figure 8 It can be seen that, with a kernel size of 3×3, the accuracy of varying channel size when the number of convolutional layers is 2 (refer to...) Figure 8 (a) Accuracy of varying channel size when the number of convolutional layers is 4 (refer to) Figure 8 (b) and the accuracy of varying channel size when the number of convolutional layers is 6 (see [reference]). Figure 8 In (c), the accuracy is highest when the momentum is 0.5.

[0078] Additionally, refer to Figure 9 It can be seen that, with a kernel size of 5×5, the accuracy of varying channel size when the number of convolutional layers is 2 (refer to...) Figure 9 (a) Accuracy of varying channel size when the number of convolutional layers is 4 (refer to) Figure 9 (b) and the accuracy of varying channel size when the number of convolutional layers is 6 (see [reference]). Figure 9 In (c), the accuracy is highest when the momentum is 0.6.

[0079] Next, the neural network device 10 can determine whether batch normalization is required (S130). That is, the neural network device 10 can determine whether the convolutional layer needs batch normalization.

[0080] For example, refer to Figure 10 With 4 convolutional layers, 9 channels, and a kernel size of 3×3, each convolutional layer may require batch normalization.

[0081] Batch standardization can be performed using the following mathematical formula 1. To independently standardize elements with scalar values, mathematical formula 1 can be used to perform operations on the n-dimensional input x = {x(1), x(2), ..., x(n)}.

[0082] [Mathematical Expression 1]

[0083]

[0084] At this point, when batch normalization is not included in each convolutional layer, refer to Figure 11 The accuracy may differ by 8%, but it is not limited to this.

[0085] Next, the neural network device 10 can determine whether a pooling layer exists (S140). That is, the neural network device 10 can determine whether a pooling layer exists in the convolutional layer.

[0086] For example, refer to Figure 10 With 4 convolutional layers, 9 channels, and a 3×3 kernel, the pooling layer can be located in the last two layers of the convolutional layers.

[0087] In other words, when the pooling layer is located in the last two layers of a convolutional layer, the reference... Figure 12 It can be seen that it has the highest accuracy rate.

[0088] On the other hand, the step S130, which determines whether batch standardization exists, can be executed after the step S140, which determines whether a pooling layer exists. However, it is not limited to this; steps S130 and S140 can be executed simultaneously.

[0089] Next, the neural network device 10 can use the optimal parameters to calculate the minimum channel size in the BNN model (S30). That is, the neural network device 10 can determine the accuracy of the minimum channel size and calculate the corresponding minimum channel size.

[0090] For example, the neural network device 10 can calculate the channel size by starting from 9 and decreasing it sequentially. In this case, the channel size may be independent of the number of convolutional layers.

[0091] Specifically, refer to Figure 13 It can be seen that with a minimum channel size of 8 and a kernel size of 3×3, the accuracy is above 96%. That is, in the BNN model, the number of stacked resistive random access memory (RRAM) elements can be minimized while maintaining or improving accuracy.

[0092] In addition, the neural network device 10 can determine the accuracy of the minimum channel size by applying the error rate of each layer according to the minimum channel size.

[0093] For example, refer to Figure 14 When the minimum channel size is 8 and the kernel size is 3×3, the accuracy is 94.069% at the 40nm node, 93.777% at the 20nm node, and 93.07% at the 10nm node.

[0094] Finally, the neural network device 10 can optimize the RRAM (resistive random access memory) layer with minimal bias using the minimum channel size (S40).

[0095] Reference Figure 15 The neural network device 10 can reduce the channel size of 50 by more than 80% while maintaining accuracy, thereby optimizing the channel size to 8.

[0096] The neural network device 10 described above may include various portable electronic communication devices for performing layer optimization methods of stacked resistive random access memory elements utilizing artificial intelligence technology. For example, as individual smart devices, it may include various terminals such as smartphones, personal digital assistants (PDAs), tablets, wearable devices (e.g., including smartwatches, smart glasses, head-mounted displays, etc.) and various Internet of Things (IoT) terminals, but is not limited thereto.

[0097] Figure 16 This is a hardware configuration diagram of an exemplary computing device that can implement the neural network device 10.

[0098] Reference Figure 16The computing device 800 may include one or more processors 810, a storage device 850 storing a computer program 851, a memory 820 loading the computer program 851 executed by the processor 810, a bus 830, and a network interface 840. However, in Figure 16 Only components relevant to embodiments of this disclosure are shown. Therefore, those skilled in the art to which this disclosure pertains will recognize that other components may also be included. Figure 16 Other general components besides those shown.

[0099] Processor 810 controls the overall operation of various configurations of computing device 800. Processor 810 may be configured as a Central Processing Unit (CPU), Microprocessor Unit (MPU), Micro Controller Unit (MCU), Graphics Processing Unit (GPU), or any type of processor known in the art of this disclosure. Additionally, processor 810 may perform operations for at least one computer program for executing a layer optimization method for stacked resistive random access memory elements utilizing artificial intelligence techniques according to embodiments of this disclosure. Computing device 800 may include one or more processors.

[0100] The memory 820 stores data supporting various functions of the computing device 800. The memory 820 stores one or more computer programs (applications, application programs, or application software) driving the computing device 800, as well as data, instructions, and information used for the operation of the computing device 800. At least a portion of the computer programs can be downloaded from an external device (not shown). Additionally, at least a portion of the computer programs may be present on the computing device 800 at the time of manufacture for use in the basic functions of the computing device 800 (e.g., receiving and sending messages).

[0101] On the other hand, in order to execute a layer optimization method for stacked resistive random access memory elements utilizing artificial intelligence technology according to embodiments of the present disclosure, memory 820 may load one or more computer programs 851 from storage device 850. Figure 16 In the example of memory 820, random access memory (RAM) is shown.

[0102] Bus 830 provides communication functionality between components of computing device 800. Bus 830 can be implemented as various types of buses, such as address bus, data bus, and control bus.

[0103] Network interface 840 supports wired and wireless network communication of computing device 800. In addition, network interface 840 can also support various communication methods other than network communication. Therefore, network interface 840 may include communication modules known in the art of this disclosure.

[0104] Storage device 850 may non-temporarily store one or more computer programs 851. Storage device 850 may include non-volatile memory such as read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, as well as hard disk, removable disk, or any form of computer-readable recording medium known in the art to which this disclosure pertains.

[0105] In addition, Figure 16 In addition to the components shown, the computing device 800 may further include an input unit and an output unit.

[0106] The input unit may include a camera for receiving image signals, a microphone for receiving audio signals, and a user input unit for receiving information from the user. The user input unit may include one or more touch keys and mechanical keys. Image data collected by the camera or audio signals collected by the microphone can be analyzed and processed according to user control commands.

[0107] The output unit outputs the command processing results in a visual, auditory, or tactile manner, and may include a display unit, a light output unit, a speaker, and a tactile output unit.

[0108] The steps of the methods or algorithms described in connection with embodiments of this disclosure can be implemented directly in hardware, or by software modules executed by hardware, or by a combination thereof. The software modules can reside in random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or any form of computer-readable recording medium known in the art to which this disclosure pertains.

[0109] The embodiments of this disclosure have been described above with reference to the accompanying drawings. However, those skilled in the art to which this disclosure pertains should understand that this disclosure can be implemented in other specific ways without changing its technical concept or essential features. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not limiting.

[0110] Industrial applicability

[0111] The disclosed technology can be applied to neural networks, neural network devices, and neural network systems that utilize stacked resistive random access memory (3D RRAM).

Claims

1. A neural network method for optimizing layers of a stacked resistive random access memory (RRAM) device using artificial intelligence (AI) technology, the method comprising the following steps: The neural network device classifies the BNN parameters in the BNN model into physical parameters and hyperparameters; The neural network device obtains optimal parameters using the physical parameters and the hyperparameters; The neural network device uses the optimal parameters to calculate the minimum channel size in the BNN model; as well as The neural network device utilizes the minimum channel size to optimize the layers of the stacked resistive switching memory elements with minimal bias.

2. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 1, wherein, The physical parameters include the number of convolutional layers, channel size, kernel size, presence of batch normalization, and presence of pooling layers. The hyperparameters include optimizer, learning rate, and momentum.

3. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 2, wherein, The optimizer includes at least one of the following algorithms: batch gradient descent, stochastic gradient descent, gradient descent, mini-batch gradient descent, momentum algorithm, adaptive gradient algorithm, root mean square propagation algorithm, and adaptive moment estimation algorithm.

4. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 3, wherein, The steps for obtaining the optimal parameters include the following: The optimal algorithm is computed using the optimizer's algorithm included in the hyperparameters; Using the learning rate included in the hyperparameters, the optimal learning rate for adjusting the weights and kernel update intensity in the BNN is calculated; and The optimal momentum is calculated by considering the momentum values ​​in the BNN using the momentum included in the hyperparameters.

5. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 4, wherein, The steps for calculating the optimal algorithm include the following: Calculate the optimal algorithm combination among at least one algorithm included in the optimizer; and Calculate the ratio of the optimal algorithm combination.

6. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 5, wherein, The optimal algorithm is, When the kernel size is 3×3, the ratio of adaptive moment estimation algorithm to stochastic gradient descent algorithm is 3:

7. When the kernel size is 5×5, the ratio of adaptive moment estimation algorithm to stochastic gradient descent algorithm is 6:

4.

7. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 4, wherein, The optimal learning rate is the same when the kernel size is 3×3 and when the kernel size is 5×5. The optimal learning rate is 0.

03.

8. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 4, wherein, The optimal momentum is, With a kernel size of 3×3, the momentum value is 0.

5. When the kernel size is 5×5, the momentum value is 0.

6.

9. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 4, wherein, The steps for obtaining the optimal parameters include the following: Determine whether the batch standardization and the pooling layer exist.

10. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 9, wherein, When the number of convolutional layers is 4, the channel size is 9, and the kernel size is 3×3, the batch normalization is included in each of the convolutional layers.

11. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 10, wherein, When the batch normalization is not included in each of the convolutional layers, the accuracy differs by 8%.

12. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 10, wherein, The batch standardization is performed using [Mathematical Formula 1]. [Mathematical Expression 1] 。 13. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 9, wherein, When the number of convolutional layers is 4, the channel size is 9, and the kernel size is 3×3, the pooling layer is located in the last two layers of the convolutional layers.

14. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 1, wherein, The steps to calculate the minimum channel size include the following: To determine the accuracy of the minimum channel size.

15. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 14, wherein, With a minimum channel size of 8, the accuracy is above 96% when the kernel size is 3×3.

16. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 15, wherein, The steps for calculating the minimum channel size further include the following steps: The accuracy of the minimum channel size is determined by applying the error rate of each layer according to the minimum channel size.

17. The neural network method for optimizing layers of stacked resistive switching memory elements using artificial intelligence technology according to claim 16, wherein, When the minimum channel size is 8 and the kernel size is 3×3 The accuracy rate is 94.069% at the 40nm node, 93.777% at the 20nm node, and 93.07% at the 10nm node.

18. A neural network system for optimizing layers of stacked resistive random access memory (RRAM) elements using artificial intelligence technology, the neural network system performing the method according to any one of claims 1 to 17 via a neural network device.

19. A computer program for executing a neural network method for optimizing layers of stacked resistive random access memory (RRAM) elements using artificial intelligence techniques, wherein, The program is combined with a computer as hardware and stored in a computer-readable recording medium to perform the method according to any one of claims 1 to 17.

Images