Artificial neural network memory system based on data locality of artificial neural networks

Abstract

An artificial neural network memory system includes at least one processor configured to generate data access requests corresponding to artificial neural network operations and at least one artificial neural network memory controller configured to record the data access requests in sequence to generate artificial neural network data locality patterns of the artificial neural network operations and generate, based on the artificial neural network data locality patterns, a speculative data access request that predicts a next data access request of the data access requests generated by the at least one processor.

Description

TECHNICAL FIELD

[0001] The disclosure relates to an artificial neural network memory system based on data locality of an artificial neural network, and more particularly, to an artificial neural network memory system in which a memory efficiently provides data required by a processor to improve the operation speed of an artificial neural network based on artificial neural network data locality of an artificial neural network model processed in units of data access requests. BACKGROUND

[0002] With the development of artificial intelligence inference capabilities, various inference services such as sound recognition, speech recognition, image recognition, object detection, driver drowsiness detection, danger moment detection, and gesture detection are mounted in various electronic devices such as artificial intelligence speakers, smartphones, smart refrigerators, VR devices, AR devices, artificial intelligence closed circuit televisions, artificial intelligence robot cleaners, tablets, notebooks, autonomous vehicles, biped robots, quadruped robots, and industrial robots.

[0003] Recently, with the development of deep learning technology, the performance of artificial neural network inference services based on big data learning has been developed. The learning and inference services of artificial neural networks repeatedly train artificial neural networks with massive learning data and infer various complex data with the trained artificial neural network model. Therefore, various services are provided to the above electronic devices by utilizing artificial neural network technology.

[0004] However, the functions and accuracy required for inference services using artificial neural networks are gradually increasing. Accordingly, the size of the artificial neural network model, the amount of calculation, and the size of the learning data are exponentially increasing. The performance required for processors and memories capable of processing inference operations of artificial neural network models is gradually increasing. In addition, artificial neural network inference services are actively provided to servers that can easily process big data based on cloud computing.

[0005] Meanwhile, edge computing using artificial neural network model technology is being actively researched. Edge computing refers to the edge or peripheral part where the computation is performed. Edge computing refers to various electronic devices that are directly generated by terminals or adjacent to terminals. Edge computing is also referred to as an edge device. The edge device can be used to immediately and reliably perform necessary tasks, such as an automatic unmanned drone, an automatic robot, or an automatic vehicle that needs to process a large amount of data within 1 / 100 seconds. Therefore, the fields to which the edge device is applied are rapidly increasing. SUMMARY

[0006] The inventors of the present disclosure have recognized that there are problems such as high power consumption, heat generation, and a bottleneck phenomenon of processor operation caused by low memory bandwidth and memory delay in the operation of a related-art artificial neural network model. Accordingly, the inventors have recognized that there are various difficulties in improving the operation processing performance of an artificial neural network model, and have recognized the need to develop an artificial neural network memory system capable of improving these problems.

[0007] Accordingly, the inventors of the present disclosure have researched an artificial neural network memory system suitable for a server system and / or edge computing. In addition, the inventors of the present disclosure have researched a neural processing unit (NPU) which is a processor of an artificial neural network memory system optimized for processing an artificial neural network model.

[0008] First, the inventors of the present disclosure have recognized that, in order to improve the calculation processing speed of an artificial neural network, it is critical to effectively control a memory in the calculation process of an artificial neural network model. The inventors of the present disclosure have recognized that, when an artificial neural network model is trained or inferred, if the memory is not properly controlled, necessary data is not prepared in advance, and thus a reduction in memory effective bandwidth and / or a delay in the supply of data of the memory can frequently occur. Further, the inventors of the present disclosure have recognized that, in this case, a starvation or an idle state in which data to be processed is not provided to a processor can occur, so that an actual operation cannot be performed, resulting in a decrease in operation performance.

[0009] Second, the inventors of the present disclosure have recognized the limitations of an artificial neural network model operation processing method at an algorithm level of the related art. For example, a prefetch algorithm of the related art is a technology that analyzes an artificial neural network model in a conceptual layer unit so that a processor reads data from a memory in each layer unit. However, the prefetch algorithm cannot recognize artificial neural network data locality existing in a word unit or a memory access request unit of an artificial neural network model at a processor-memory level (i.e., a hardware level). The inventors of the present disclosure have recognized that it is difficult to optimize data transmission / reception operation at a processor-memory level only by a prefetch technology.

[0010] Third, the inventors of the present disclosure have recognized "artificial neural network data locality" which is a unique feature of an artificial neural network model. The inventors of the present disclosure have recognized that artificial neural network data locality exists in a word unit or a memory access request unit at a processor-memory level, and that effective memory bandwidth is maximized and a delay in data provided to a processor is minimized by utilizing artificial neural network data locality, thereby improving artificial neural network learning / inference operation processing performance of the processor.

[0011] Specifically, the "artificial neural network data locality" of the artificial neural network model recognized by the inventors of the present disclosure refers to the sequence information of the word units of the data required for the processor to perform the calculation processing of the artificial neural network, which is performed according to the structure and operation algorithm of the artificial neural network model when the processor processes a specific artificial neural network model. In addition, the inventors of the present disclosure have recognized that the artificial neural network data locality is maintained for the operation of the iterative learning and / or inference of the artificial neural network model provided to the processor in the operation processing sequence of the artificial neural network model. Accordingly, the inventors of the present disclosure have recognized that the processing sequence of the data required for the artificial neural network operation processed by the processor is maintained in the word unit and provides or analyzes the information to be used for the artificial neural network operation while maintaining the artificial neural network data locality. In other words, the word unit of the processor can refer to an element unit as a basic unit to be processed by the processor. For example, when the neural network processing unit processes the multiplication of N-bit input data and M-bit kernel weight, the input data word unit of the processor can be N bits, and the weight data word unit can be M bits. In addition, the inventors of the present disclosure have recognized that the word unit of the processor can be set differently according to the layer, feature map, kernel, activation function, etc. of the artificial neural network model, respectively. Accordingly, the inventors of the present disclosure have also recognized that an accurate memory control technique is necessary for the operation in the word unit.

[0012] The inventors of the present disclosure have recognized that artificial neural network data locality is constructed when an artificial neural network model is compiled by a compiler to be executed in a specific processor. In addition, the inventors have recognized that artificial neural network data locality can be constructed according to operational characteristics of an algorithm applied to the compiler and the artificial neural network model as well as the processor. In addition, the inventors of the present disclosure have recognized that artificial neural network data locality of an artificial neural network model to be processed can be constructed in various forms depending on a computational method of the artificial neural network model of the processor, such as feature map tiling, a smoothing technique of a processing element, a number of processing elements of the processor, a cache capacity of a feature map, such as a weight, in the processor, a memory hierarchy in the processor, and an algorithm characteristic of a compiler that determines a sequence of computational operations of the processor to compute the artificial neural network model, even in the same artificial neural network model. This is because, even if the same artificial neural network model is computed, the sequence of data required in each time in a clock unit is different due to the above factors. That is, the inventors of the present disclosure have recognized that the sequence of data necessary for computation of the artificial neural network model is conceptually a computational sequence of layers, unit convolutions, and / or matrix multiplications of the artificial neural network. Also, the inventors of the present disclosure have recognized that, in the sequence of data required for physical computation, artificial neural network data locality of the artificial neural network model is constructed in a word unit at a processor-memory level, i.e., a hardware level. In addition, the inventors of the present disclosure have recognized that artificial neural network data locality depends on the processor and the compiler for the processor.

[0013] Fourth, the inventors of the present disclosure have recognized that, when an artificial neural network memory system constructed to be provided with artificial neural network data locality information to utilize artificial neural network data locality is provided, processing performance of an artificial neural network model can be maximized at a processor-memory level.

[0014] The inventors of the present disclosure have recognized that, when an artificial neural network memory system accurately computes a word unit of artificial neural network data locality of an artificial neural network model, the processor also finds operation processing sequence information of the word unit to be a minimum unit for the processor to process the artificial neural network model. That is, the inventors of the present disclosure have recognized that, when an artificial neural network memory system that utilizes artificial neural network data locality is provided, the artificial neural network memory system can accurately predict whether specific data is to be read from a memory at a specific time to be provided to the processor or whether the specific data is to be computed by the processor to be stored in the memory in a word unit at a specific time. Accordingly, the inventors of the present disclosure have recognized that an artificial neural network system is provided to prepare data to be requested by the processor in advance in a word unit.

[0015] In other words, the inventors of the present disclosure have recognized that if the artificial neural network memory system knows the artificial neural network data locality, then when the processor calculates the convolution of the particular input data and the particular kernel using a technique such as feature map tiling, the operation processing sequence of the convolution processed as the kernel moves in a particular direction is also known in the word unit.

[0016] That is, it is recognized that the artificial neural network memory system predicts which data is necessary for the processor by exploiting the artificial neural network data locality, thereby predicting the memory read / write operations to be requested by the processor and the data to be processed by the processor is prepared in advance to minimize or eliminate the increase in memory effective bandwidth and / or the data supply delay of the memory. Further, the inventors have recognized that the starvation or idle state of the processor can be minimized when the artificial neural network memory system provides the data to be processed by the processor with the necessary timing. Accordingly, the inventors of the present disclosure have recognized that the operation processing performance can be improved and the power consumption can be reduced by the artificial neural network memory system.

[0017] Fifth, the inventors of the present disclosure have recognized that even without providing the artificial neural network data locality information to the artificial neural network memory controller, after the artificial neural network memory controller is disposed in the communication channel between the processor processing the artificial neural network model and the memory, the data access request to the memory is analyzed when the processor processes the operation of the particular artificial neural network model to infer the artificial neural network data locality of the artificial neural network model being processed by the processor in the data access request unit between the processor and the memory. That is, the inventors of the present disclosure have recognized that each artificial neural network model has a unique artificial neural network data locality, so that the processor generates a particular sequence of data access requests according to the artificial neural network data locality at the processor-memory level. Further, the inventors of the present disclosure have recognized that based on the fact that the artificial neural network data locality is maintained while the processor iteratively processes the learning / inference operation of the artificial neural network model, the access queue of the data stored in the memory is used for the data request between the processor and the memory.

[0018] Accordingly, the inventors of the present disclosure dispose the artificial neural network memory controller in the communication channel of the processor operating the artificial neural network model and the memory. Further, the inventors observe the data access request between the processor and the memory for one or more learning and inference operations, recognizing that the artificial neural network memory controller can infer the artificial neural network data locality in the data access request unit. Accordingly, the inventors of the present disclosure have recognized that the artificial neural network data locality can be inferred by the artificial neural network memory controller even without providing the artificial neural network data locality information.

[0019] Accordingly, the inventors of the present disclosure have recognized that, based on artificial neural network data locality reconstructed in a data access request unit, memory read / write operations to be requested by a processor can be predicted, and an increase in memory effective bandwidth and / or a delay in memory data supply can be minimized or substantially eliminated by preparing data to be processed by the processor in advance. Further, the inventors of the present disclosure have recognized that, when an artificial neural network memory system provides data to be processed by a processor with necessary timing, a processor starvation or idle state occurrence rate can be minimized.

[0020] Accordingly, an object to be achieved by the present disclosure is to provide an artificial neural network memory system that optimizes artificial neural network operation of a processor by exploiting artificial neural network data locality of an artificial neural network model operating at a processor-memory level.

[0021] Accordingly, an object to be achieved by the present disclosure is to provide an artificial neural network memory system including an artificial neural network memory controller that detects data access requests generated by a processor to generate a data locality pattern of an artificial neural network model being processed by the processor, thereby improving a latency problem of a memory by preparing data access requests to be requested by the processor in advance. However, the present disclosure is not limited thereto, and other objects apparent to those skilled in the art can be understood from the following description.

[0022] An artificial neural network memory system according to an exemplary embodiment of the present disclosure is provided. The system can be configured to include at least one processor configured to generate data access requests corresponding to artificial neural network operations, and at least one artificial neural network memory controller configured to sequentially record the data access requests to generate an artificial neural network data locality pattern of the artificial neural network operations, and generate an advance data access request predicting a next data access request of the data access requests generated by the at least one processor based on the artificial neural network data locality pattern.

[0023] An artificial neural network memory system according to an exemplary embodiment of the present disclosure can be configured to include at least one processor configured to process an artificial neural network model, and at least one artificial neural network memory controller configured to store artificial neural network data locality information of the artificial neural network model, and predict data to be requested by the at least one processor according to the artificial neural network data locality information to generate an advance data access request.

[0024] The artificial neural network memory system can be configured to further include at least one memory and a system bus configured to control communication of the artificial neural network memory controller, the at least one processor, and the at least one memory.

[0025] According to an exemplary embodiment of the disclosure, the artificial neural network memory system includes a processor, a memory, and a cache memory, and can be configured to generate a pre-data access request including data to be requested by the processor based on artificial neural network data locality information, and store data corresponding to the pre-data access request from the memory in the cache memory before the request of the processor.

[0026] According to an exemplary embodiment of the disclosure, the artificial neural network memory system can operate in any one of a first mode and a second mode to predict artificial neural network data locality information, wherein the first mode is configured to operate by receiving artificial neural network data locality information, and the second mode is configured to operate by observing data access requests generated by the processor.

[0027] Here, the artificial neural network data locality can be artificial neural network data locality reconstructed at a processor-memory level.

[0028] The at least one artificial neural network memory controller can be configured to further sequentially generate the pre-data access request based on the artificial neural network data locality pattern.

[0029] The at least one artificial neural network memory controller can be configured to generate the pre-data access request before generating a next data access request.

[0030] The at least one processor can be configured to transmit a data access request to the at least one artificial neural network memory controller.

[0031] The at least one artificial neural network memory controller can be configured to output the pre-data access request in response to the data access request.

[0032] The data access request can be configured to further include a memory address.

[0033] The data access request can be configured to further include a start address and / or an end address of the memory.

[0034] The at least one artificial neural network memory controller can be configured to generate a memory access request based on one of a data access request generated by the at least one processor and a pre-data access request generated by the artificial neural network memory controller.

[0035] The data access request can be configured to further include a start address of the memory and a contiguous data read (or burst read) trigger.

[0036] The data access request can be configured to further include a start address of the memory and a quantity of contiguous data information.

[0037] The data access request and the advanced data access can be configured to further include a data access request token of the same matching memory address.

[0038] The data access request can be configured to further include identification information to identify whether it is a memory read command or a write command.

[0039] The data access request can be configured to further include identification information to identify whether it is a memory overwrite command.

[0040] The data access request can be configured to further include identification information to identify whether it is inference data, weight data, or feature map data.

[0041] The data access request can be configured to further include identification information to identify whether it is learning data or evaluation data.

[0042] The data access request can be configured to further include identification information to identify whether the artificial neural network operation is an operation for learning or an operation for inference.

[0043] When the at least one processor generates a next data access request, the at least one artificial neural network memory controller can be configured to determine whether the advanced data access request and the next data access request are the same request.

[0044] When the advanced data access request and the next data access request are the same request, the at least one artificial neural network memory controller can be configured to maintain an artificial neural network data locality pattern.

[0045] When the advanced data access request and the next data access request are different, the at least one artificial neural network memory controller can be configured to update the artificial neural network data locality pattern.

[0046] The artificial neural network data locality pattern can be configured to further include data in which memory addresses of the data access requests are sequentially recorded.

[0047] The at least one artificial neural network memory controller can be configured to generate the artificial neural network data locality pattern by detecting a repetitive pattern of memory addresses included in the data access requests.

[0048] The artificial neural network data locality pattern can be configured by memory addresses having a repetitive cycle characteristic.

[0049] The artificial neural network data locality pattern can be configured to further include identification information for identifying a start and an end of an operation of the artificial neural network model.

[0050] The at least one processor can be configured to be provided with data corresponding to a data access request from the artificial neural network memory controller.

[0051] The at least one artificial neural network memory controller can be configured to further include an artificial neural network model configured to machine learn the artificial neural network data locality pattern.

[0052] The at least one artificial neural network memory controller can be configured to store an updated pattern and a previous pattern of the artificial neural network data locality pattern to determine whether the artificial neural network model is changed.

[0053] The at least one artificial neural network memory controller can be configured to determine whether the data access request is a request for one artificial neural network model or a mixture of requests for a plurality of artificial neural network models.

[0054] When there are a plurality of artificial neural network models, the at least one artificial neural network memory controller can be configured to further generate artificial neural network data locality patterns corresponding to the number of artificial neural network models.

[0055] The at least one artificial neural network memory controller can be configured to individually generate a corresponding early data access request based on the artificial neural network data locality pattern.

[0056] The at least one artificial neural network memory controller can be configured to further generate a memory access request corresponding to the data access request.

[0057] The at least one artificial neural network memory controller can be configured to further generate a memory access request corresponding to the early data access request.

[0058] Each of the data access request, the early data access request, and the memory access request can be configured to include a corresponding memory address value and an operation mode, respectively.

[0059] The at least one artificial neural network memory controller can be configured to further generate a memory access request including at least a part of information included in the data access request and the early data access request.

[0060] The at least one memory can be configured to communicate with the at least one artificial neural network memory controller, and the at least one memory can be configured to operate in response to a memory access request output from the at least one artificial neural network memory controller.

[0061] The at least one memory can be configured to store at least one of inference data, weight data, and feature map data.

[0062] The at least one artificial neural network memory controller can be configured to further include a cache memory configured to store data transmitted from the at least one memory in response to the memory access request.

[0063] When the at least one processor outputs a next data access request, the at least one artificial neural network memory controller determines whether the previous data access request and the next data access request are the same request. If the previous data access request and the next data access request are the same, the at least one artificial neural network memory controller provides data stored in the cache memory to the at least one processor, and if the previous data access request and the next data access request are not the same, the at least one artificial neural network memory controller generates a new memory access request according to the next data access request.

[0064] The at least one artificial neural network memory controller sequentially generates the at least one memory access request based on a remaining capacity of the cache memory to minimize the remaining capacity of the cache memory.

[0065] The at least one artificial neural network memory controller can be configured to measure an effective bandwidth of the at least one memory in response to the memory access request.

[0066] The at least one artificial neural network memory controller can be configured to be provided with a necessary bandwidth of the at least one memory in response to the memory access request.

[0067] The at least one artificial neural network memory controller can be configured to measure an inference per second (IPS) number of artificial neural network operations by calculating a number of repetitions of an artificial neural network data locality pattern within a certain time.

[0068] The at least one artificial neural network memory controller can be configured to calculate a time taken to repeat an artificial neural network data locality pattern once and a data size for calculating an effective bandwidth required for an artificial neural network operation.

[0069] The at least one memory can further include a DRAM including a refresh function to update a voltage of a memory cell, and the at least one artificial neural network memory controller can be configured to selectively control a refresh of a memory address region of the at least one memory corresponding to a memory access request corresponding to the early data access request.

[0070] The at least one memory can further include a pre-charge function to charge a global bit line of the memory with a specific voltage, and the at least one artificial neural network memory controller can be configured to selectively provide a pre-charge to a memory address region of the at least one memory corresponding to a memory access request corresponding to the early data access request.

[0071] The at least one memory can further include a plurality of memories, and the at least one artificial neural network memory controller can be configured to measure an effective bandwidth of the plurality of memories, respectively.

[0072] The at least one memory can further include a plurality of memories, and the at least one artificial neural network memory controller can be configured to measure a latency of the plurality of memories, respectively.

[0073] The at least one memory can further include a plurality of memories, and the at least one artificial neural network memory controller can be configured to divide and store data to be stored in the plurality of memories based on the effective bandwidth and the latency of each of the plurality of memories.

[0074] The data is composed of a group of L bits, and the plurality of memories includes a first memory and a second memory. The first memory is configured to divide and store M bits of data from the group of L bits based on a first effective bandwidth or a first latency, and the second memory is configured to divide and store N bits of data from the group of L bits based on a second effective bandwidth or a second latency. A sum of the M bits and the N bits can be configured to be less than or equal to the L bits.

[0075] The plurality of memories can further include a third memory, and the third memory can be configured to store O bits of data from the group of L bits based on a third effective bandwidth or a third latency. A sum of the M bits, the N bits, and the O bits can be configured to be equal to the L bits.

[0076] The at least one artificial neural network memory controller can be configured to further include a cache memory configured to merge and store the data divided to be stored in the plurality of memories.

[0077] The data is composed of P data packets, and the plurality of memories includes a first memory and a second memory. The first memory is configured to store R data packets of the P data packets based on a first effective bandwidth or a first latency, and the second memory is configured to store S data packets of the P data packets based on a second effective bandwidth or a second latency. A sum of R and S can be configured to be less than or equal to P.

[0078] The plurality of memories can further include a third memory, and the third memory can be configured to store T data packets from the P data packets based on a third effective bandwidth or a third latency. A sum of R, S, and T can be configured to be equal to P.

[0079] The at least one memory can further include a plurality of memories, and the at least one artificial neural network memory controller can further include a cache memory and can be configured to merge data distributed to be stored in the plurality of memories to store the merged data in the cache memory.

[0080] The at least one memory can further include a plurality of memories, and the at least one artificial neural network memory controller can be configured to store division information of data divided to be stored in the plurality of memories.

[0081] The at least one artificial neural network memory controller can be configured to store a portion of data as much as a latency in the cache memory based on an early data access request and a latency value of the at least one memory.

[0082] The at least one artificial neural network memory controller can be configured to store a portion of data in the cache memory based on an early data access request and a required data bandwidth of the at least one memory.

[0083] When the at least one processor generates a next data access request, the at least one artificial neural network memory controller first provides data stored in the cache memory and controls remaining data from the at least one memory in a read burst mode to reduce a latency of the at least one memory.

[0084] When the at least one processor generates a next data access request based on an early data access request and a latency value of the at least one memory, the at least one artificial neural network memory controller starts in a read burst mode of the at least one memory to reduce a latency of the at least one memory.

[0085] A system bus configured to control communication of the artificial neural network memory controller, the at least one processor, and the at least one memory can be further included.

[0086] The at least one artificial neural network memory controller can be configured to have a master right of the system bus.

[0087] The at least one artificial neural network memory controller can further include an artificial neural network model. The artificial neural network model can be machine trained to raise a control right of a system bus to be higher than a case where the early data access request is not generated, when the early data access request is generated.

[0088] The at least one artificial neural network memory controller can be configured to secure an effective bandwidth of the system bus until at least one memory completes a memory access request.

[0089] The at least one artificial neural network memory controller can be configured to calculate a specific bandwidth required for the system bus to process a specific memory access request based on the artificial neural network data locality pattern, and the at least one artificial neural network memory controller can be configured to control an effective bandwidth of the system bus based on the specific bandwidth.

[0090] The at least one artificial neural network memory controller can be disposed in the system bus, and the system bus can be configured to dynamically change a bandwidth of the system bus based on the artificial neural network data locality pattern generated in the system bus.

[0091] The at least one artificial neural network memory controller can operate to process an artificial neural network operation preferentially in a processing time of a memory access request, and to process an operation other than the artificial neural network operation at other times.

[0092] The at least one artificial neural network memory controller and the at least one processor can be configured to directly communicate with each other.

[0093] The artificial neural network memory controller can be configured to further include a first access queue which is an access queue dedicated to an artificial neural network operation, and a second access queue which is an access queue other than the artificial neural network operation, and the artificial neural network memory controller can be configured to select an access queue to provide data according to a priority setting.

[0094] The at least one artificial neural network memory controller can further include a plurality of hierarchical cache memories, and the at least one artificial neural network memory controller can be configured to further include an artificial neural network model configured to perform machine learning of a data access request between layers of the plurality of hierarchical cache memories.

[0095] The at least one artificial neural network memory controller can be configured to further be provided with at least one of effective bandwidth, power consumption, and delay information of each layer of the plurality of hierarchical cache memories.

[0096] At least one artificial neural network memory system includes at least one processor configured to generate a data access request corresponding to an artificial neural network operation, at least one artificial neural network memory controller configured to store an artificial neural network data locality pattern of an artificial neural network operation generated from a compiler and generate an early data access request predicting a next data access request of the data access request generated by the at least one processor based on the artificial neural network data locality pattern, and at least one memory configured to communicate with the at least one artificial neural network memory controller. The at least one memory can be configured to operate according to a memory access request output from the at least one artificial neural network memory controller.

[0097] At least one artificial neural network memory system can be configured to further include at least one memory and a system bus configured to control communication of the artificial neural network memory controller, the at least one processor, and the at least one memory.

[0098] At least one artificial neural network memory controller can be disposed in the system bus, and the at least one artificial neural network memory controller can be configured to raise a control authority of the system bus to be higher than a control authority when there is no memory access request until the at least one memory completes a response to the memory access request.

[0099] At least one artificial neural network memory controller includes one or more artificial neural network memory controllers configured to be included in a DRAM.

[0100] At least one artificial neural network memory controller includes one or more artificial neural network memory controllers configured to be included in at least one processor.

[0101] At least one memory can further include a DRAM or the at least one memory can be a DRAM, and the at least one artificial neural network memory controller can be configured to re-adjust an access queue of the memory access request.

[0102] The artificial neural network memory controller can be configured to generate at least one access queue.

[0103] At least one memory can include the artificial neural network memory controller, and the artificial neural network memory controller can be configured to separately generate an access queue dedicated to the artificial neural network operation.

[0104] At least one artificial neural network memory controller can be configured to re-adjust an access queue of the memory access request.

[0105] The at least one memory can further include a read burst function, and the at least one artificial neural network memory controller can be configured to set a storage area of the at least one memory in consideration of the read burst function.

[0106] The at least one memory can further include a read burst function, and the at least one artificial neural network memory controller can be configured to set a storage area of the at least one memory in consideration of the read burst function.

[0107] The at least one processor can include a plurality of processors, and the at least one artificial neural network memory controller can be configured to set a priority of a data access request of a processor processing an artificial neural network operation higher than a data access request of a processor processing an operation other than the artificial neural network operation among the plurality of processors.

[0108] According to an exemplary embodiment of the disclosure, in a system processing an artificial neural network, a delay of a memory providing data to a processor can be substantially eliminated or reduced by artificial neural network data locality.

[0109] According to an exemplary embodiment of the disclosure, an artificial neural network memory controller can prepare data of an artificial neural network model to be processed at a processor-memory level before being requested by a processor.

[0110] According to an exemplary embodiment of the disclosure, learning and inference operation processing time of an artificial neural network model processed by a processor is shortened to improve operation processing performance of the processor and improve power efficiency of operation processing at a system level.

[0111] Effects according to the disclosure are not limited to the above examples, and various effects are included in the specification. BRIEF DESCRIPTION OF DRAWINGS

[0112] Figure 1A is a schematic block diagram for explaining a processor and an artificial neural network memory controller of an artificial neural network memory system based on artificial neural network data locality according to an exemplary embodiment of the disclosure.

[0113] Figure 1B is a schematic diagram illustrating an example of an exemplary neural network processing unit for explaining a reconstruction of an artificial neural network data locality pattern applicable to various exemplary embodiments of the disclosure.

[0114] Figure 2 is a schematic diagram for explaining an artificial neural network data locality pattern according to an exemplary embodiment of the disclosure.

[0115] Figure 3This is a schematic diagram illustrating an exemplary artificial neural network model for explaining patterns of artificial neural network data locality that can be applied to various exemplary embodiments of this disclosure.

[0116] Figure 4 This is an exemplary embodiment of the present disclosure used to explain the analysis via an artificial neural network memory controller. Figure 3 A schematic diagram of the data locality pattern of artificial neural networks generated by the artificial neural network model of A.

[0117] Figure 5 It is an explanation and Figure 4 A schematic diagram of tokens and recognition information corresponding to the data locality pattern of artificial neural networks.

[0118] Figure 6 This is a schematic diagram illustrating an exemplary embodiment of the present disclosure used to explain advance data access requests and next data access requests generated by an artificial neural network memory controller based on the data locality pattern of an artificial neural network.

[0119] Figure 7 This is a flowchart illustrating the operation of an artificial neural network memory controller according to an exemplary embodiment of the present disclosure.

[0120] Figure 8 This is a schematic block diagram illustrating an artificial neural network memory system according to another exemplary embodiment of the present disclosure.

[0121] Figure 9 This is a schematic diagram illustrating the operation of a memory system based on a comparative embodiment of the present disclosure.

[0122] Figure 10 This is a schematic diagram illustrating a memory system according to another exemplary embodiment of the present disclosure.

[0123] Figure 11 This is a schematic block diagram illustrating an artificial neural network memory system according to yet another exemplary embodiment of this disclosure.

[0124] Figure 12 This is a schematic diagram illustrating exemplary identification information used to interpret data access requests.

[0125] Figure 13 This is a schematic diagram used to explain the energy consumption per unit operation of an artificial neural network memory system.

[0126] Figure 14 This is a schematic diagram illustrating various exemplary embodiments of the present disclosure for explaining an artificial neural network memory system. Detailed Implementation

[0127] The advantages and features of the present disclosure and methods of achieving the advantages and features will become apparent from a consideration of the following detailed description taken in conjunction with the accompanying drawings. However, the present disclosure is not limited to the exemplary embodiments disclosed herein but is susceptible to various modifications. The exemplary embodiments are provided so that the present disclosure can be thoroughly and completely understood and the scope of the present disclosure can be easily drawn, by those skilled in the art. Accordingly, the present disclosure will be defined only by the scope of the claims.

[0128] For the convenience of description, the detailed description of the present disclosure can be described with reference to the accompanying drawings using specific exemplary embodiments in which the present disclosure can be implemented as an example. Although the components of various exemplary embodiments of the present disclosure are different from each other, the manufacturing method, the operation method, the algorithm, the shape, the process, the structure, and the features described in a specific exemplary embodiment can be combined with or included in other embodiments. In addition, it should be understood that the position or arrangement of each constituent element in each disclosed exemplary embodiment can be changed without departing from the spirit and scope of the present disclosure. The features of various embodiments of the present disclosure can be partially or entirely combined with or combined with each other, and can be interlocked and operated in various ways that can be understood by those skilled in the art, and the embodiments can be executed independently of each other or in association with each other.

[0129] The shapes, sizes, ratios, angles, numbers, etc. shown in the drawings for describing the exemplary embodiments of the present disclosure are merely examples, and the present disclosure is not limited thereto. Throughout the specification, the same reference numerals denote the same elements. Also, in the following description, detailed explanations of known related technologies will be omitted to avoid unnecessarily obscuring the subject matter of the present disclosure. The terms such as "include", "have", and "consist of" used herein are generally intended to allow the addition of other components, unless the terms are used with the term "only". Unless otherwise explicitly stated, any singular cited can include plural. Even if not explicitly stated, components are interpreted to include ordinary error ranges. When the positional relationship of two elements is described using, for example, "on", "above", "below", "close to", or "adjacent to", one component can be located between the two components, unless these terms are used with the term "immediately" or "directly", when an element or layer is disposed "on" another element or layer, another layer or another element can be directly inserted on another element or between them.

[0130] Figure 1A is a schematic block diagram of a processor and an artificial neural network memory controller of an artificial neural network memory system for explaining artificial neural network data locality based on an artificial neural network according to an exemplary embodiment of the present disclosure.

[0131] Reference Figure 1AThe artificial neural network memory system 100 can be configured to include at least one processor 110 and at least one artificial neural network memory controller 120. That is, at least one processor 110 according to an exemplary embodiment of the disclosure is provided, and a plurality of processors can be used, and at least one artificial neural network memory controller 120 according to an exemplary embodiment of the disclosure is provided, and a plurality of artificial neural network memory controllers can be used.

[0132] Hereinafter, for the convenience of description, when the at least one processor 110 is one processor, it can be referred to as the processor 110.

[0133] Hereinafter, for the convenience of description, when the at least one artificial neural network memory controller 120 is one artificial neural network memory controller 120, it can be referred to as the artificial neural network memory controller 120.

[0134] The processor 110 is configured to process an artificial neural network model. For example, the processor 110 processes inference of an artificial neural network model trained to perform a specific inference function to provide an inference result of the artificial neural network model according to input data. For example, the processor 110 processes learning of an artificial neural network model for performing a specific inference function to provide a trained artificial neural network model. The specific inference function can include various inference functions that can be inferred by an artificial neural network, such as object recognition, speech recognition, image processing.

[0135] The processor 110 can be configured to include at least one of a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), a digital signal processing device (DSP), an arithmetic and logic unit (ALU), and an artificial neural network processing unit (NPU). However, the processor 110 of the disclosure is not limited to the above-described processor.

[0136] The processor 110 can be configured to communicate with the artificial neural network memory controller 120. The processor 110 can be configured to generate a data access request. The data access request can be transmitted to the artificial neural network memory controller 120. Here, the data access request can refer to a request to access data required for the processor 110 to process inference or learning of an artificial neural network model.

[0137] The processor 110 can transmit a data access request to the artificial neural network memory controller 120 to be provided with data required for inference or learning of an artificial neural network model from the artificial neural network memory controller 120 or to provide the artificial neural network memory controller 120 with an inference or learning result of an artificial neural network processed by the processor 110.

[0138] The processor 110 can provide an inference result or a learning result obtained by processing a specific artificial neural network model. At this time, the processor 110 can be configured to process operations of the artificial neural network for inference or learning in a specific sequence.

[0139] The processor 110 needs to process operations of the artificial neural network in a specific sequence because each artificial neural network model is configured to have a unique artificial neural network structure. That is, each artificial neural network model is configured to have unique artificial neural network data locality according to the unique artificial neural network structure. In addition, the sequence of operations of the artificial neural network model processed by the processor 110 is determined according to the unique artificial neural network data locality.

[0140] In other words, when the artificial neural network model is compiled by the compiler to be executed in the specific processor, the artificial neural network data locality can be configured. The artificial neural network data locality can be configured according to an algorithm applied to the compiler and the artificial neural network model and the operation characteristics of the processor.

[0141] The artificial neural network model to be processed by the processor 110 can be compiled by the processor 110 and the compiler that considers the algorithm characteristics of the artificial neural network model. That is, when the driving characteristics of the processor 110 are known with the structure and algorithm information of the artificial neural network model, the compiler can be configured to provide the artificial neural network data locality information to the artificial neural network memory controller 120 in the order of the word unit.

[0142] For example, the weight values of a specific layer of a specific artificial neural network model at the algorithm level of the layer unit computing technique can be calculated. However, the weight values of the specific layer of the specific artificial neural network model at the processor-memory level according to the exemplary embodiment of the present disclosure can be calculated in the word unit planned to be processed by the processor 110.

[0143] For example, when the size of the cache memory of the processor 110 is smaller than the data size of the weights of the specific layer of the artificial neural network model to be processed, the processor 110 can be compiled not to process the weight values of the specific layer at a time.

[0144] That is, when the processor 110 calculates the weight values and node values of the specific layer, the space of the cache memory storing the result values can be insufficient because the weight values are too large. In this case, the data access request generated by the processor 110 can be increased to a plurality of data access requests. Therefore, the processor 110 can be configured to process the increased data access requests in a specific sequence. In this case, the sequence of operations at the algorithm level and the sequence of operations according to the artificial neural network data locality at the processor-memory level can be different from each other.

[0145] That is, by considering hardware characteristics of a processor and a memory, an algorithm-level artificial neural network operation sequence can be reconfigured by processor-memory level artificial neural network data locality to process a corresponding artificial neural network model.

[0146] Artificial neural network data locality existing in an artificial neural network model at a processor-memory level can be defined as information that predicts an operation order of an artificial neural network model to be processed by a processor 110 at a processor-memory level based on a data access request order requested by the processor 110 to the memory.

[0147] In other words, even in the same artificial neural network model, artificial neural network data locality of an artificial neural network model can be variously configured according to an operation function of the processor 110, such as a feature map tiling technique or a fixing technique of a processing element, a cache capacity, the number of processing elements of the processor 110, a feature map and a weight in the processor 110, a memory hierarchy structure in the processor 110, and an algorithm characteristic of a compiler that determines a sequence of a computing operation of the processor 110 to compute an artificial neural network model.

[0148] For example, the feature map tiling technique is an artificial neural network technique that divides a convolution and as a convolution region is divided, a feature map is divided to be computed. Thus, due to tiling a convolution, even the same artificial neural network model has different artificial neural network data locality.

[0149] For example, the fixing technique is a technique that controls a driving method of a processing element PE in a neural network processing unit. According to the fixing technique, one of a data type to be processed, such as an input feature map, a weight, and an output feature map, is fixed to a processing element to be reused. Thus, a type of data or sequence requested by the processor 110 to the memory can vary.

[0150] That is, even in the same artificial neural network model, artificial neural network data locality can be reconfigured according to various algorithms and / or techniques. Thus, artificial neural network data locality can be wholly or partially reconfigured by various conditions, such as a processor, a compiler, or a memory.

[0151] Figure 1B FIG. 1 is a schematic diagram illustrating an example of an exemplary neural network processing unit to explain a reconfiguration of an artificial neural network data locality pattern applicable to various exemplary embodiments of the present disclosure.

[0152] Referring to FIG. 1, Figure 1B illustrates an exemplary fixing technique applicable when the processor 110 is a neural network processing unit NPU.

[0153] The processing elements PE can be configured in an array and each processing element can be configured to include a multiplier (x) and an adder (+). The processing elements PE can be connected to a buffer memory or cache memory, such as a global buffer. The processing elements PE can fix one of an input feature map pixel (Ifmap pixel: I), a filter weight W, and a partial sum (Psum: P) to a register of the processing element PE. The remaining data can be provided as input data of the processing element PE. When accumulation of the partial sum P is completed, it can become an output feature map pixel.

[0154] Figure 1B (a) in FIG. illustrates a weight stationary (WS) technique. According to the weight stationary (WS) technique, the filter weights W0 to W7 are fixed to respective register files of the processing elements PE, and input feature map pixels I that are input in parallel to the processing elements PE are moved from the 0th input feature map pixel I0 to the eighth input feature map pixel I8 to perform operations. The partial sums P0 to P8 can be accumulated in the processing elements PE connected in series. The partial sums P0 to P8 can be sequentially moved to subsequent processing elements. All multiplication and accumulation (MAC) operations using the fixed filter weights W0 to W7 need to be mapped to the same processing elements PE for serial processing.

[0155] According to the above-described configuration, during convolution operations of the filter weights W in the register files, reuse of the filter weights W is maximized to minimize access energy consumption of the filter weights W.

[0156] It is noted that, since the weight stationary WS technique is applied to the artificial neural network model in the compiling step, artificial neural network data locality of the reconstructed artificial neural network model is optimized for the weight stationary WS technique at the processor-memory level. For example, according to the weight stationary WS technique, the filter weights W0 to W7 can be preferentially stored in the processing elements PE for efficiency of operations. Accordingly, the artificial neural network data locality can be reconstructed in order of the filter weights W, the input feature map pixels I, and the partial sums P, so that a data access request sequence to be generated by the processor 110 can be determined according to the reconstructed artificial neural network data locality.

[0157] Figure 1B(b) illustrates an output stationary (OS) technique. According to the output stationary (OS) technique, partial sums P0 to P7 are fixed to respective register files of processing elements PE to be accumulated, and filter weights W input to the processing elements PE in parallel are moved from a 0th input filter weight W0 to a 7th filter weight W7 to perform an operation. Input feature pixels I0 to I7 can be moved to the processing elements PE connected in series. Each partial sum P0 to P7 needs to be fixed to each processing element PE to be mapped to perform a multiply and accumulate (MAC) operation.

[0158] According to the above-described configuration, during a convolution operation of the filter weights W in the processing elements PE, the partial sums P are fixed to the register files of the processing elements PE to maximize reuse of the partial sums P and minimize energy consumption according to movement of the partial sums P. When accumulation of the fixed partial sums P is completed, they can become output feature maps.

[0159] It should be noted that, when the processor 110 applies the output stationary (OS) technique, artificial neural network data locality of the artificial neural network model is reconfigured to be optimized at a processor-memory level for the output stationary (OS) technique. For example, according to the output stationary (OS) technique, partial sums P0 to P7 are preferentially stored in the processing elements PE for efficiency of an operation. Thereby, artificial neural network data locality can be reconfigured in order of the partial sums P, the filter weights W, and the input feature pixels I, so that a data access request sequence generated by the processor 110 can be determined according to the reconfigured artificial neural network data locality. The artificial neural network model compiler receives hardware characteristic information of the processor 110 and the memory to convert it into code in which the artificial neural network model operates at a processor-memory level. At this time, the artificial neural network model is converted into code executed by the processor, so that the artificial neural network model can be converted into a low-level code.

[0160] That is, according to the above-described factors, even if the same artificial neural network model is processed, the processor 110 can change an order of data required at each moment in a clock unit. Accordingly, artificial neural network data locality of the artificial neural network model can be configured to be different at a hardware level.

[0161] However, when configuration of the artificial neural network data locality is completed, an operation order of the processor 110 and a data processing order required for the operation can be accurately repeated at each learning operation or inference operation of the corresponding artificial neural network model.

[0162] Hereinafter, the artificial neural network memory system 100 according to the exemplary embodiment of the disclosure can be configured to predict the next data to be requested by the processor 110 based on the accurate operation sequence provided by the artificial neural network data locality to improve the memory latency problem and the memory bandwidth problem, thereby improving the operation processing performance of the artificial neural network, reducing power consumption.

[0163] The artificial neural network memory controller 120 according to the exemplary embodiment of the disclosure is configured to be provided with artificial neural network data locality information of an artificial neural network model to be processed by the processor 110, or is configured to analyze the artificial neural network data locality of an artificial neural network model being processed by the processor 110.

[0164] The artificial neural network memory controller 120 can be configured to receive a data access request generated by the processor 110.

[0165] The artificial neural network memory controller 120 can be configured to monitor or record the data access request received from the processor 110. The artificial neural network memory controller 120 observes the data access request output by the processor 110 which is processing the artificial neural network to accurately predict the data access queue that will be requested later. One data access request can be configured to include at least one word unit data.

[0166] The artificial neural network memory controller 120 can be configured to sequentially record or monitor the data access request received from the processor 110.

[0167] The data access request recorded by the artificial neural network memory controller 120 can be stored in various forms, such as a log file, a table, or a list. However, the artificial neural network memory controller 120 according to the exemplary embodiment of the disclosure is not limited to the type or formant of the record of the data access request.

[0168] The data access request monitored by the artificial neural network memory controller 120 can be stored in any memory in the artificial neural network memory controller 120. However, the artificial neural network memory controller 120 according to the exemplary embodiment of the disclosure is not limited to the monitoring method of the data access request.

[0169] The artificial neural network memory controller 120 can be configured to further include any memory for recording or monitoring the data access request. However, the artificial neural network memory controller 120 according to the exemplary embodiment of the disclosure is not limited thereto and can be configured to communicate with an external memory.

[0170] The artificial neural network memory controller 120 can be configured to monitor or record data access requests received from the processor 110 to analyze the data access requests.

[0171] That is, the artificial neural network memory controller 120 can be configured to analyze the received data access requests to analyze the artificial neural network data locality of the artificial neural network model being processed by the processor 110.

[0172] That is, the artificial neural network memory controller 120 can be configured to analyze the artificial neural network data locality of the artificial neural network model compiled to operate at the processor-memory level.

[0173] That is, the artificial neural network memory controller 120 can be configured to analyze the operation processing order of the artificial neural network in units of memory access requests generated by the processor based on the processor-memory level artificial neural network data locality of the artificial neural network model to analyze the artificial neural network data locality of the artificial neural network model.

[0174] According to the above-described configuration, the artificial neural network memory controller 120 can analyze the artificial neural network data locality restructured at the processor-memory level.

[0175] In some example embodiments, the compiler can be configured to analyze the artificial neural network data locality of the artificial neural network model in a word unit.

[0176] In some example embodiments, at least one artificial neural network memory controller can be configured to provide the artificial neural network data locality analyzed by the compiler in a word unit. Here, the word unit can vary to 8 bits, 16 bits, 32 bits, 64 bits, etc. according to the word unit of the processor 110. Here, the word unit can be set to a different word unit, e.g., 2 bits, 3 bits, or 5 bits, according to a quantization algorithm of a kernel, a feature map, etc. of the compiled artificial neural network model.

[0177] The artificial neural network memory controller 120 can be configured to include a special function register. The special function register can be configured to store artificial neural network data locality information.

[0178] The artificial neural network memory controller 120 can be configured to operate in different modes according to whether the artificial neural network data locality information is stored.

[0179] If the artificial neural network memory controller 120 stores the artificial neural network data locality information, the artificial neural network memory controller 120 can predict in order the data processing sequence of the artificial neural network model to be processed by the processor 110 in a word unit in advance, so that the artificial neural network memory controller 120 can be configured not to record a separate data access request. However, it is not limited thereto, and the artificial neural network memory controller 120 can be configured to verify whether the stored artificial neural network data locality is erroneous when comparing the stored artificial neural network data locality information and the data access request generated by the processor.

[0180] If the artificial neural network memory controller 120 is not provided with the artificial neural network data locality information, the artificial neural network memory controller 120 can be configured to observe the data access request generated by the processor 110 to operate in a mode in which the artificial neural network data locality of the artificial neural network model processed by the processor 110 is predicted.

[0181] In some exemplary embodiments, the artificial neural network memory system can be configured to include a processor, a memory, and a cache memory, and to generate an advance data access request including data to be requested by the processor based on artificial neural network data locality information. The artificial neural network memory system can be configured to store data corresponding to the advance data access request from the memory in the cache memory before the request of the processor. At this time, the artificial neural network memory system can be configured to operate in any one of a first mode configured to operate by receiving the artificial neural network data locality information and a second mode configured to operate by observing the data access request generated by the processor to predict the artificial neural network data locality information. According to the above-described configuration, when the artificial neural network memory system is provided with the artificial neural network data locality information, the data to be requested by the processor is predicted and prepared in a word unit in advance. In addition, even if the artificial neural network data locality information is not provided, the data access request generated by the processor is monitored for a predetermined period of time to predict the artificial neural network data locality in the data access request unit in which the processor is processing. In addition, even if the artificial neural network data locality information is provided, the artificial neural network memory system independently monitors the data access request to reconstruct the artificial neural network data locality to verify the provided artificial neural network data locality. Accordingly, a change or an error of the artificial neural network model can be sensed.

[0182] In some exemplary embodiments, the at least one artificial neural network memory controller and the at least one processor can be configured to directly communicate with each other. According to the above-described configuration, the artificial neural network memory controller can directly receive a data access request from the processor, so that a delay caused by a system bus between the processor and the artificial neural network memory controller can be eliminated. In other words, for the direct communication of the processor and the artificial neural network memory controller, a dedicated bus can be further included, or a dedicated communication channel can be further included, but is not limited thereto.

[0183] In some exemplary embodiments, the artificial neural network data locality information can be configured to be selectively stored in the processor 110 and / or the artificial neural network memory controller 120. The artificial neural network data locality information can be configured to be stored in a special function register included in the processor 110 and / or the artificial neural network memory controller 120. However, it is not limited thereto, and the artificial neural network data locality information can be configured to be stored in any memory or register that can communicate with the artificial neural network memory system.

[0184] Figure 2 is a schematic diagram for explaining an artificial neural network data locality pattern according to an exemplary embodiment of the disclosure. Hereinafter, the artificial neural network data locality and the artificial neural network data locality pattern of the artificial neural network model will be described with reference to Figure 2 The artificial neural network data locality and the artificial neural network data locality pattern of the artificial neural network model will be described with reference to

[0185] The artificial neural network memory controller 120 is configured to sequentially record or monitor data access requests received from the processor 110.

[0186] The artificial neural network memory controller 120 is configured to generate an artificial neural network data locality pattern including data locality of the artificial neural network model being processed by the processor 110. That is, the artificial neural network memory controller 120 can be configured to analyze data access requests associated with the artificial neural network model generated by the processor 110 to generate a specific pattern of repetition. That is, when the data access request is observed, the artificial neural network data locality information can be stored as the artificial neural network data locality pattern.

[0187] Referring to Figure 2 , eighteen data access requests are sequentially recorded in the artificial neural network memory controller 120. Each data access request is configured to include identification information.

[0188] The identification information included in the data access request can be configured to include various information.

[0189] For example, the identification information can be configured to include at least a memory address value and an operation mode value.

[0190] For example, the memory address value can be configured to include a memory address value corresponding to the requested data, but the disclosure is not limited thereto.

[0191] For example, the memory address value can be configured to include a start value and an end value of a memory address corresponding to the requested data. According to the above configuration, it is considered that data is sequentially stored between the start value and the end value of the memory address. Accordingly, it is possible to reduce the capacity for storing the memory address value.

[0192] For example, the memory address value can be configured to include a start value of a memory address corresponding to the requested data and a data continuous read trigger value. According to the above configuration, data can be continuously read from the start value of the memory address until the continuous read trigger value is changed. According to the above configuration, data can be continuously read, and thus it is possible to increase the memory effective bandwidth.

[0193] For example, the memory address value can be configured to include a start value of a memory address corresponding to the requested data and information on the number of data. The unit of the number of data can be determined based on a unit of memory capacity. For example, the unit can be one of 1 byte of 8 bits, 1 word of 4 bytes, 1 block of 1024 bytes, but the disclosure is not limited thereto. According to the above configuration, data can be continuously read from the start value of the memory address as many as the number of data of the set unit size. According to the above configuration, data can be continuously read, and thus it is possible to increase the memory effective bandwidth.

[0194] For example, when the memory is a non-volatile memory, the memory address value can further include a physical logical address mapping table or flash translation layer information, but the disclosure is not limited thereto.

[0195] For example, the operation mode can be configured to include a read mode and a write mode.

[0196] For example, the operation mode can be configured to further include overwrite, but the disclosure is not limited thereto.

[0197] The artificial neural network memory controller 120 can be configured to determine whether the identification information of each data access request is the same.

[0198] For example, the artificial neural network memory controller 120 can be configured to determine whether the memory address and the operation mode of each data access request are the same. In other words, the artificial neural network memory controller 120 can be configured to detect a data access request value having the same memory address value and the same operation mode.

[0199] For example, when the memory address value and the operation mode of the first data access request are the same as the memory address value and the operation mode of the tenth data access request, the artificial neural network memory controller 120 is configured to generate an artificial neural network data locality pattern corresponding to the corresponding memory address value and operation mode.

[0200] The artificial neural network data locality pattern is configured to include data in which the memory addresses of the data access requests are sequentially recorded.

[0201] That is, the artificial neural network memory controller 120 can be configured to detect a repetition cycle of data access requests having the same memory address value and operation mode to generate an artificial neural network data locality pattern configured by data access requests having repeated memory address values and operation modes.

[0202] That is, the artificial neural network memory controller 120 can be configured to generate an artificial neural network data locality pattern by detecting a repetition pattern of memory addresses included in the data access requests.

[0203] Referring to Figure 2 When the artificial neural network memory controller 120 identifies that the memory address value and the operation mode of the first data access request are the same as the memory address value and the operation mode of the tenth data access request, the artificial neural network memory controller 120 can be configured to generate an artificial neural network data locality pattern from the early data access request of the starting data access request to the repeated data access request in the same data access request. In this case, the artificial neural network memory controller 120 can be configured to generate an artificial neural network data locality pattern including the first data access request to the ninth data access request.

[0204] That is, referring to Figure 2 The artificial neural network data locality pattern described can be configured to include the memory address value and the operation model value in the order of the first data access request, the second data access request, the third data access request, the fourth data access request, the fifth data access request, the sixth data access request, the seventh data access request, the eighth data access request, and the ninth data access request.

[0205] The artificial neural network data locality pattern generated by the artificial neural network memory controller 120 can be stored in various forms (e.g., a log file, a table, or a list), and the artificial neural network memory controller 120 according to an exemplary embodiment of the disclosure is not limited to the recording type or format of the artificial neural network data locality pattern.

[0206] The artificial neural network data locality pattern generated by the artificial neural network memory controller 120 can be stored in any memory of the artificial neural network memory controller 120, and the artificial neural network memory controller 120 according to an exemplary embodiment of the disclosure is not limited to the structure or method of the memory storing the artificial neural network data locality pattern.

[0207] The artificial neural network memory controller 120 can be configured to further include any memory for storing the artificial neural network data locality pattern. However, the artificial neural network memory controller 120 according to an exemplary embodiment of the disclosure is not limited thereto and can be configured to communicate with an external memory.

[0208] That is, the artificial neural network memory system 100 according to an exemplary embodiment of the disclosure can be configured to include at least one processor 110 configured to generate a data access request corresponding to an artificial neural network operation, and an artificial neural network memory controller 120 configured to sequentially record the data access request to generate an artificial neural network data locality pattern.

[0209] When the artificial neural network memory controller 120 generates the artificial neural network data locality pattern, the artificial neural network memory controller 120 can be configured to determine whether the memory address value and the operation mode value of the data access request received from the processor 110 match any one of the memory address value and the operation mode value included in the previously generated artificial neural network data locality pattern.

[0210] Referring to Figure 2 When the artificial neural network memory controller 120 receives the tenth data access request from the processor 110, the artificial neural network memory controller 120 can be configured to determine whether the received data access request has the same memory address value as the memory address value contained in the artificial neural network data locality pattern.

[0211] Referring to Figure 2 When the artificial neural network memory controller 120 receives the tenth data access request, the artificial neural network memory controller 120 can be configured to detect that the start value [0] and the end value [0x1000000] of the memory address value of the tenth data access request are the same as the start value [0] and the end value [0x1000000] of the memory address value of the first data access request, and detect that the read mode value of the operation mode of the tenth data access request is the same as the read mode value of the operation mode of the first data access request, to determine that the tenth data access request is the same as the first data access request and that the tenth data access request is an artificial neural network operation.

[0212] When the artificial neural network memory controller 120 receives the eleventh data access request, the artificial neural network memory controller 120 can be configured to detect that the start value [0x1100000] and the end value [0x1110000] of the memory address value of the eleventh data access request are the same as the start value [0x1100000] and the end value [0x1110000] of the memory address value of the second data access request, and detect that the write mode value of the operation mode of the eleventh data access request is the same as the write mode value of the operation mode of the second data access request, to determine that the eleventh data access request is the same as the second data access request, and that the eleventh data access request is the artificial neural network operation.

[0213] Referring again to Figure 2 , the artificial neural network memory controller 120 does not generate the artificial neural network data locality pattern from the first data access request to the ninth data access request. In this case, the artificial neural network memory controller 120 is initialized, or the processor 110 does not perform the artificial neural network operation. Accordingly, the artificial neural network memory controller 120 does not detect a match of the pattern with the ninth data access request. The artificial neural network memory controller 120 can determine the identity of the first data access request at the tenth data access request, generate the artificial neural network data locality pattern, and record whether the pattern matches. The tenth data access request to the eighteenth data access request are the same as the first data access request to the ninth data access request, so the artificial neural network memory controller 120 can determine that the pattern of the tenth data access request to the eighteenth data access request matches the artificial neural network data locality pattern.

[0214] That is, the artificial neural network memory controller 120 can be configured to determine whether an operation being processed by the processor 110 is the artificial neural network operation by utilizing the artificial neural network data locality pattern. According to the above-described configuration, even if the artificial neural network memory controller 120 receives only the data access request including the memory address value and the operation mode value generated by the processor 110, the artificial neural network memory controller 120 can determine that the processor 110 is processing the artificial neural network operation. Accordingly, the artificial neural network memory controller 120 can determine whether the processor 110 is currently performing the artificial neural network operation based on the artificial neural network data locality pattern, without separate additional identification information.

[0215] As will be referred to with reference to Figure 2As further described, each data access request can be configured to be stored as a token. For example, data access requests for each artificial neural network can be tokenized for storage. For example, data access requests for each artificial neural network can be tokenized based on identification information. For example, data access requests for each artificial neural network can be tokenized based on memory address values. However, exemplary embodiments of the present disclosure are not limited thereto, and the token can be referred to as a code, an identifier, or the like.

[0216] For example, the first data access request can be stored as token [1]. The fourth data access request can be stored as token [4]. The seventh data access request can be stored as token [7]. For example, the artificial neural network data locality pattern can be stored as token [1-2-3-4-5-6-7-8-9]. For example, the tenth data access request has the same memory address value and the same operation mode value as token [1], and thus the tenth data access request can be stored as token [1]. The thirteenth data access request has the same memory address value and the same operation mode value as token [4], and thus the thirteenth data access request can be stored as token [4]. Accordingly, when the artificial neural network memory controller 120 detects a token identical to the token of the artificial neural network data locality pattern, the artificial neural network memory controller can be configured to determine that the corresponding data access request is an artificial neural network operation.

[0217] According to the above-described configuration, the artificial neural network memory controller 120 can easily and quickly identify and distinguish data access requests by utilizing the tokenized artificial neural network data locality pattern. Furthermore, even when additional identification information and / or data are further added in the data access request, the artificial neural network memory controller uses the same token to utilize the token, even when the additional information of the data access request is increased to easily and quickly identify and distinguish data access requests.

[0218] In some exemplary embodiments, the artificial neural network data locality pattern stored in the artificial neural network memory controller can be eliminated or initialized. For example, when the artificial neural network data locality pattern is not used when it expires at a predetermined time, for example, when no data access request matching the artificial neural network data locality pattern is generated at a certain time, the artificial neural network memory controller determines that the artificial neural network data locality pattern is less frequently used to eliminate or initialize the artificial neural network data locality pattern.

[0219] According to the above-described configuration, the availability of the storage space of the memory in which the artificial neural network data locality pattern is stored can be improved.

[0220] In some example embodiments, the artificial neural network memory controller can be configured to store the update pattern and the previous pattern of the artificial neural network data locality pattern to determine whether the artificial neural network model is changed. That is, when there are a plurality of artificial neural network models, the artificial neural network memory controller can be configured to further generate the artificial neural network data locality pattern corresponding to the number of artificial neural network models.

[0221] For example, when the first artificial neural network data locality pattern is the token [1-2-3-4-5-6-7-8-9] and the second artificial neural network data locality pattern is the token [11-12-13-14-15-16], if the processor generates a data access request corresponding to the token [1], the artificial neural network memory controller can be configured to select the first artificial neural network data locality pattern. Alternatively, if the processor generates a data access request corresponding to the token

[11] , the artificial neural network memory controller can be configured to select the second artificial neural network data locality pattern.

[0222] According to the above-described configuration, the artificial neural network memory controller can store a plurality of artificial neural network data locality patterns, and when the artificial neural network model processed by the processor is changed to another artificial neural network model, the previously stored artificial neural network data locality pattern can be quickly applied.

[0223] In some example embodiments, the artificial neural network memory controller can be configured to determine whether the data access request is a request for one artificial neural network model or a mixture of requests for a plurality of artificial neural network models. In addition, the artificial neural network memory controller can be configured to predict data access requests corresponding to artificial neural network data locality for each of the plurality of artificial neural network models.

[0224] For example, the processor can simultaneously process a plurality of artificial neural network models, and in this case, the data access request generated by the processor can be a mixed data access request corresponding to the plurality of artificial neural network models.

[0225] For example, when the first artificial neural network data locality pattern is the token [1-2-3-4-5-6-7-8-9] and the second artificial neural network data locality pattern is the token [11-12-13-14-15-16], the processor 110 can generate tokens corresponding to data access requests in the order of [1-11-2-3-12-13-14-4-5-6-15-16-7-8-9].

[0226] The artificial neural network memory controller knows each artificial neural network data locality pattern, so that even if the token [1] is generated, and then the token

[11] is generated, the artificial neural network memory controller can predict that the token [2] will be generated next. Accordingly, the artificial neural network memory controller can generate an advanced data access request corresponding to the token [2]. Also, even if the token [2] is generated after the token

[11] is generated, the artificial neural network memory controller can predict that the token

[12] will be generated next. Accordingly, the artificial neural network memory controller can generate an advanced data access request corresponding to the token

[12] .

[0227] According to the above-described configuration, the artificial neural network memory controller 120 predicts data access requests to be generated by the processor 110 processing a plurality of artificial neural network models for each artificial neural network model, to predict and prepare data to be requested by the processor 110.

[0228] In some example embodiments, the artificial neural network memory controller can be configured to store a plurality of artificial neural network data locality patterns.

[0229] For example, when the processor processes two artificial neural network models, the artificial neural network memory controller can be configured to store an artificial neural network data locality pattern for each artificial neural network model.

[0230] According to the above-described configuration, when processing operations of each artificial neural network model, the next data access request corresponding to each model can be predicted, so that the processing speed of artificial neural network operations can be improved according to example embodiments of the disclosure.

[0231] In some example embodiments, the artificial neural network memory controller can be configured to further include an artificial neural network model configured to machine-learn artificial neural network data locality patterns.

[0232] According to the above-described configuration, the artificial neural network model of the artificial neural network memory controller can be configured to perform reinforcement learning in real time on data access requests generated by the processor. Further, the artificial neural network model of the artificial neural network memory controller can be a model trained by using artificial neural network data locality patterns of artificial neural network models well known in the relevant field as learning data. Accordingly, the artificial neural network memory controller can extract artificial neural network data locality patterns from various artificial neural network models. Specifically, when a plurality of artificial neural network models are processed by requests of a plurality of users (e.g., servers), such an approach can be effective.

[0233] As will be referred to with reference to Figure 2As described elsewhere, the artificial neural network memory controller 120 can be configured to dynamically and in real time monitor the artificial neural network model processed by the processor 110 and determine whether the artificial neural network model has changed.

[0234] For example, the artificial neural network memory controller 120 can be configured to statistically determine the reliability of the artificial neural network data locality patterns using the pattern matching frequency of the artificial neural network data locality patterns. It can be configured such that the reliability of the artificial neural network data locality patterns increases as the pattern matching frequency increases, and the reliability of the artificial neural network data locality patterns decreases as the pattern matching frequency decreases.

[0235] According to the above configuration, when the processor 110 repeatedly processes a specific artificial neural network model, the artificial neural network memory controller 120 can improve the predictive reliability of the artificial neural network data locality of the specific artificial neural network model.

[0236] Figure 3 This is a schematic diagram illustrating an exemplary artificial neural network model for explaining patterns of data locality applicable to various exemplary embodiments of this disclosure.

[0237] Currently being made by Figure 3 The exemplary artificial neural network model 1300 processed by the processor 110 shown can be any artificial neural network model trained to perform a specific reasoning function. For the sake of description, an artificial neural network model in which all nodes are fully connected has been shown, but this disclosure is not limited thereto.

[0238] although Figure 3 Not shown, but applicable to the artificial neural network model of this disclosure, may be a convolutional neural network (CNN), which is a type of deep neural network (DNN). Exemplary artificial neural network models may be, for example, fully convolutional networks (FCNs) with VGG, VGG16, DenseNET and encoder-decoder structures, deep neural networks (DNNs) such as SegNet, DeconvNet, DeepLABV3+, or U-net, or SqueezeNet, AlexNet, ResNet18, MobileNet-v2, GoogLeNet, ResNet-v2, ResNet50, ResNet101 and Inception-v3, or ensemble models based on at least two different models, but the artificial neural network model of this disclosure is not limited thereto.

[0239] The above exemplary artificial neural network model can be configured to have artificial neural network data locality.

[0240] Referring again to Figure 3 , the artificial neural network data locality of the artificial neural network model processed by the processor 110 is described in detail.

[0241] The exemplary artificial neural network model 1300 includes an input layer 1310, a first connection network 1320, a first hidden layer 1330, a second connection network 1340, a second hidden layer 1350, a third connection network 1360, and an output layer 1370.

[0242] The connection network of the artificial neural network has a corresponding weight value. The weight value of the connection network is multiplied by the input node value, and the cumulative value of the multiplied value is stored in the node of the corresponding output layer.

[0243] In other words, the connection network of the artificial neural network model 1300 is represented by a line, and the weight is represented by a symbol .

[0244] In other words, various activation functions can be additionally provided to impart nonlinearity to the cumulative value. The activation function can be, for example, a sigmoid function, a hyperbolic tangent function, or a ReLU function, but the present disclosure is not limited thereto.

[0245] The input layer 1310 of the exemplary artificial neural network model 1300 includes input nodes x1 and x2.

[0246] The first connection network 1320 of the exemplary artificial neural network model 1300 includes a connection network having six weight values, which connects the nodes of the input layer 1310 and the nodes of the first hidden layer 1330.

[0247] The first hidden layer 1330 of the exemplary artificial neural network model 1300 includes nodes a1, a2, and a3. The weight values of the first connection network 1320 are multiplied by the node values of the corresponding input layer 1310, and the cumulative value of the multiplied value is stored in the first hidden layer 1330.

[0248] The second connection network 1340 of the exemplary artificial neural network model 1300 includes a connection network having nine weight values, which connects the nodes of the first hidden layer 1330 and the nodes of the second hidden layer 1350.

[0249] The second hidden layer 1350 of the exemplary artificial neural network model 1300 includes nodes b1, b2, and b3. The weight values of the second connection network 1340 are multiplied by the node values of the corresponding first hidden layer 1330, and the cumulative value of the multiplied value is stored in the second hidden layer 1350.

[0250] The third connection network 1360 of the exemplary artificial neural network model 1300 includes a connection network having six weight values connecting nodes of the second hidden layer 1350 and nodes of the output layer 1370.

[0251] The output layer 1370 of the exemplary artificial neural network model 1300 includes nodes y1 and y2. The weight values of the third connection network 1360 are multiplied by the corresponding input node values of the second hidden layer 1350, and the accumulated values of the multiplied values are stored in the output layer 1370.

[0252] According to the structure of the artificial neural network model 1300 described above, it is recognized that the operations of each layer need to be sequentially performed. That is, there can be a problem in that, when the structure of the artificial neural network model is determined, the order of operations of each layer needs to be determined, and when the operations are performed in different orders, the inference result can not be accurate. The order of operations or the order of data flow according to the structure of the artificial neural network model can be defined as artificial neural network data locality.

[0253] In addition, for convenience of description, in Figure 2 , although the layer unit is described, the exemplary embodiments of the disclosure are not limited to the layer unit. The processor 110 according to the exemplary embodiments of the disclosure processes data based on artificial neural network data locality, so that the processor can operate in a word unit or in a data access request unit, rather than in a layer unit. Here, the data size of the data access request can be less than or equal to the data size of the corresponding layer.

[0254] Referring again to Figure 3 , for example, for the multiplication operation of the weight values of the first connection network 1320 and the node values of the input layer 1310, the processor 110 can generate a data access request in a layer unit.

[0255] However, the layer operation of the weight values of the first connection network 1320 and the node values of the input layer 1310 is not processed as one data access request, but can be processed as a plurality of divided sequential data access requests according to the feature map division convolution of the processor 110, the fixed technology of the processing element, the number of processing elements of the processor, the cache capacity of the processor 110, the memory hierarchy structure of the processor 110, and / or the compiler algorithm of the processor 110.

[0256] When the data access request to be requested by the processor 110 is divided into a plurality of data access requests, the order of the data access requests requested to be divided can be determined by the artificial neural network data locality. At this time, the artificial neural network memory controller 120 can be configured to have the artificial neural network data locality to prepare to provide data corresponding to the next data access request to be requested by the processor 110. Alternatively, the artificial neural network network memory controller 120 can be configured to predict the artificial neural network data locality which will be prepared to provide data corresponding to the next data access request to be requested by the processor 110.

[0257] The data access requests and the artificial neural network data locality generated by the processor 110 during the artificial neural network operation of the artificial neural network model 1300 shown in FIG. 13 will be described. Figure 3

[0258] The processor 110 generates a first data access request to read the input node values of the input layer 1310 of the artificial neural network model 1300. The first data access request includes a first memory address value and a read mode value. The first data access request can be stored as a token [1].

[0259] Next, the processor 110 generates a second data access request to read the weight values of the first connection network 1320 of the artificial neural network model 1300. The second data access request includes a second memory address value and a read mode value. The second data access request can be stored as a token [2].

[0260] Next, the processor 110 generates a third data access request for storing the node values of the first hidden layer 1330, which are obtained by multiplying the weight values of the first connection network 1320 of the artificial neural network model 1300 by the node values of the input layer 1310 and accumulating the results. The third data access request includes a third memory address value and a write mode value. The third data access request can be stored as a token [3].

[0261] Next, the processor 110 generates a fourth data access request to read the node values stored in the first hidden layer 1330 of the artificial neural network model 1300. The fourth data access request includes a third memory address value and a read mode value. The fourth data access request can be stored as a token [4].

[0262] Next, the processor 110 generates a fifth data access request to read the weight values of the second connection network 1340 of the artificial neural network model 1300. The fifth data access request includes a fifth memory address value and a write mode value. The fifth data access request can be stored as a token [5].

[0263] ​Next, the processor 110 generates a sixth data access request for storing node values of the second hidden layer 1350 obtained by multiplying and accumulating the weight values of the second connection network 1340 of the artificial neural network model 1300 with the node values of the first hidden layer 1330. The sixth data access request includes a sixth memory address value and a write mode value. The sixth data access request can be stored as a token [6].

[0264] Next, the processor 110 generates a seventh data access request to read the node values stored in the second hidden layer 1350 of the artificial neural network model 1300. The seventh data access request includes the sixth memory address value and a read mode value. The seventh data access request can be stored as a token [7].

[0265] Next, the processor 110 generates an eighth data access request to read the weight values of the third connection network 1360 of the artificial neural network model 1300. The eighth data access request includes an eighth memory address value and a read mode value. The eighth data access request can be stored as a token [8].

[0266] Next, the processor 110 generates a ninth data access request for storing node values of the output layer 1370 obtained by multiplying and accumulating the weight values of the third connection network 1360 of the artificial neural network model 1300 with the node values of the second hidden layer 1350. The ninth data access request includes a ninth memory address value and a write mode value. The ninth data access request can be stored as a token [9]. The node values can be feature maps, activation maps, etc., but are not limited thereto. The weight values can be kernel windows, but are not limited thereto.

[0267] That is, the processor 110 needs to generate the first to ninth data access requests for inference of the exemplary artificial neural network model 1300. If the sequence of data access requests generated by the processor 110 is mixed, the artificial neural network data locality of the artificial neural network model 1300 is broken, which can cause an error or affect the accuracy of the inference result of the artificial neural network model 1300. For example, the processor 110 can calculate the second layer first and then calculate the first layer. Accordingly, the processor 110 can be configured to sequentially generate data access requests based on artificial neural network data locality. Accordingly, assuming that the artificial neural network memory controller 120 can sequentially generate data access requests based on artificial neural network data locality when the processor 110 operates the artificial neural network.

[0268] However, as described above, each data access request can be reinterpreted at a processor-memory level according to hardware characteristics of the processor. In the above example, it has been described that the available capacity of the cache memory of the processor is sufficient, and the data size of the node value and the data size of the weight value are smaller than the available capacity of the cache memory. Accordingly, it is described that each layer is processed in one data access request unit. If the data size of the artificial neural network model such as the weight value, the feature map, the kernel, the activation map, etc. is greater than the available capacity of the cache memory of the processor, the corresponding data access request can be divided into a plurality of data access requests, and in this case, the artificial neural network data locality of the artificial neural network model can be reconstructed.

[0269] The artificial neural network memory controller 120 according to an exemplary embodiment of the disclosure can generate an artificial neural network data locality pattern so that the artificial neural network memory controller can operate to correspond to the artificial neural network data locality of the artificial neural network model to be actively processed by the processor.

[0270] That is, even if the actual artificial neural network data locality of the artificial neural network model being processed by the processor 110 is not known, the artificial neural network memory controller 120 can actually analyze the artificial neural network data locality by analyzing the recorded data access requests.

[0271] That is, even if the structure information of the artificial neural network model being processed by the processor 110 is not provided, the artificial neural network memory controller 120 can actually analyze the artificial neural network data locality by analyzing the recorded data access requests.

[0272] In some exemplary embodiments, the artificial neural network memory controller can be configured to have an artificial neural network data locality pattern generated in advance at a processor-memory level.

[0273] Figure 4 is a schematic diagram illustrating an artificial neural network data locality pattern generated by analyzing an artificial neural network model by an artificial neural network memory controller according to an exemplary embodiment of the disclosure. Figure 3 is a schematic diagram illustrating an artificial neural network data locality pattern generated by analyzing an artificial neural network model by an artificial neural network memory controller according to an exemplary embodiment of the disclosure. Figure 5 is a schematic diagram illustrating a token and identification information corresponding to the artificial neural network data locality pattern. Figure 4 is a schematic diagram illustrating a token and identification information corresponding to the artificial neural network data locality pattern.

[0274] Figure 4 The artificial neural network data locality pattern 1400 shown is displayed as a token only for the convenience of description. Referring to FIG. 14, the artificial neural network data locality pattern 1400 is generated by analyzing the artificial neural network model 1401. Figures 1A to 4The artificial neural network data locality pattern 1400 of the artificial neural network model 1300 is stored as a token [1-2-3-4-5-6-7-8-9]. Identification information of a token corresponding to the artificial neural network data locality pattern 1400 shown in FIG. 14A is shown in FIG. 14B. Figure 5

[0275] Each data access request is configured to include identification information. Each data access request can be expressed in a token, but for the convenience of description, the disclosure is not limited to a token.

[0276] According to the artificial neural network data locality pattern 1400, the artificial neural network memory controller 120 can sequentially predict the order of the tokens to be generated after the current token.

[0277] For example, the artificial neural network data locality pattern 1400 can be configured to have a circular type pattern in which the order is connected from the last token to the start token, but the disclosure is not limited thereto.

[0278] For example, the artificial neural network data locality pattern 1400 can be configured by a memory address having a repetitive circular characteristic, but the disclosure is not limited thereto.

[0279] For example, the artificial neural network data locality pattern 1400 can be configured to further include identification information for identifying the start and end of the operation of the artificial neural network model, but the disclosure is not limited thereto.

[0280] For example, the start and end of the artificial neural network data locality pattern 1400 can be configured to be distinguished as a start token and a last token of the pattern, but the disclosure is not limited thereto.

[0281] According to the above configuration, when the processor 110 repeatedly inferences a specific artificial neural network model, since the artificial neural network data locality pattern 1400 is a circular type pattern, even if the current inference of the specific artificial neural network model ends, the start of the next inference can be predicted.

[0282] For example, in the case where the artificial neural network model recognizes an image object of a front camera installed in an autonomous vehicle at a speed of 30 IPS (inferences per second), the same inference is repeatedly in a specific cycle. Accordingly, when the above-described circular type artificial neural network data locality pattern is utilized, the repeated data access request can be predicted.

[0283] ​When the identification information is additionally described as an example, the token [3] and the token [4] of the artificial neural network data locality pattern 1400 have the same memory address value but have different operation modes. Accordingly, even though the memory address value is the same, the operation mode is different, so that the artificial neural network memory controller 120 can be configured to classify the third data access request and the fourth data access request as different tokens. However, the identification information of the exemplary embodiment of the present disclosure is not limited to the operation mode, but can be configured to predict the artificial neural network data locality pattern only with the memory address value.

[0284] The artificial neural network memory controller 120 can be configured to generate a corresponding early data access request based on the artificial neural network data locality pattern 1400.

[0285] The artificial neural network memory controller 120 can be configured to further sequentially generate an early data access request based on the artificial neural network data locality pattern 1400.

[0286] According to the above-described configuration, when the processor 110 generates a specific data access request included in the artificial neural network data locality pattern 1400, the artificial neural network memory controller 120 can predict at least one data access request sequentially after the specific data access request. For example, when the processor 110 generates the token [1], the artificial neural network memory controller 120 can predict that a data access request corresponding to the token [2] is subsequently generated. For example, when the processor 110 generates the token [3], the artificial neural network memory controller 120 can predict that a data access request corresponding to the token [4] is subsequently generated. For example, when the processor 110 generates the token [1], the artificial neural network memory controller 120 can predict that data access requests corresponding to the tokens [2-3-4-5-6-7-8-9] are sequentially generated.

[0287] In other words, when the processor 110 processes a plurality of artificial neural network models, a data locality pattern that is not predicted can intervene between the tokens of the artificial neural network data locality pattern 1400. For example, after the token [2], a new token [4] can be interrupted. However, even in this case, the artificial neural network memory controller 120 can predict and prepare for the processor 110 to generate the token [3] after the token [2].

[0288] For example, when the processor 110 generates the token [9], the artificial neural network memory controller 120 can predict that the processor 110 generates the token [1].

[0289] Figure 6is a schematic diagram for explaining a pre-data access request and a next data access request generated based on artificial neural network data locality patterns by an artificial neural network memory controller according to an exemplary embodiment of the present disclosure.

[0290] The artificial neural network memory controller 120 according to an exemplary embodiment of the present disclosure can be configured to predict a next data access request to be subsequently requested by the processor 110 using artificial neural network data locality patterns to generate a pre-data access request.

[0291] Referring to Figure 6 , the data access request token refers to a token corresponding to a data access request received by the artificial neural network memory controller 120 from the processor 110. The pre-data access request token refers to a token corresponding to a data access request obtained by predicting, by the artificial neural network memory controller 120, a data access request to be subsequently requested by the processor 110 based on artificial neural network data locality patterns. The next data access request token is a token of a data access request actually generated by the processor 110 after the pre-data access request token is generated, but the token of the present disclosure is only an example for convenience of description, and thus the present disclosure is not limited to the token.

[0292] The data access request and the pre-data access request can correspond to the data access request token. In this case, the data access request and the pre-data access request matching the specific data access request token can be configured to have the same memory address. That is, the data access request and the pre-data access request can be configured to include the same memory address.

[0293] For example, when the data access request token is [3] and the pre-data access request token is [3], the memory address value of each token can be the same. That is, the data access request and the pre-data access request can be configured to include the same operation mode value. For example, when the data access request token is [3] and the pre-data access request token is [3], the operation mode value of each token can be the same.

[0294] Referring to Figure 6 When the processor 110 generates a data access request corresponding to the token [1], the artificial neural network memory controller 120 generates a pre-data access request corresponding to the token [2]. The processor 110 generates a next data access request corresponding to the token [2] after the pre-data access request is generated. The artificial neural network memory controller 120 is configured to determine whether the pre-data access request accurately predicts the next data access request. The same token corresponds to the pre-data access request and the next data access request, and thus the artificial neural network memory controller 120 can determine pattern matching.

[0295] Next, for example, when the processor 110 generates a data access request corresponding to the token [2], the artificial neural network memory controller 120 generates an early data access request corresponding to the token [3]. The processor 110 generates a next data access request corresponding to the token [3] after generating the early data access request. The artificial neural network memory controller 120 is configured to determine whether the early data access request accurately predicts the next data access request. The same token corresponds to the early data access request and the next data access request, so that the artificial neural network memory controller 120 can determine the pattern matching.

[0296] For example, when the processor 110 generates a data access request corresponding to the token [9], the artificial neural network memory controller 120 generates an early data access request corresponding to the token [1]. The processor 110 generates a next data access request corresponding to the token [9] after generating the early data access request. The artificial neural network memory controller 120 is configured to determine whether the early data access request accurately predicts the next data access request. The same token corresponds to the early data access request and the next data access request, so that the artificial neural network memory controller 120 can determine the pattern matching.

[0297] When the processor 110 generates a next data access request after the artificial neural network memory controller 120 generates an early data access request, the artificial neural network memory controller 120 can be configured to determine whether the early data access request and the next data access request are the same request.

[0298] According to the above-described configuration, the artificial neural network memory system 100 can detect a change in artificial neural network data locality of the artificial neural network model processed by the processor 110. Accordingly, even when the artificial neural network model changes, the artificial neural network memory controller 120 can analyze the changed artificial neural network data locality.

[0299] When the artificial neural network memory controller 120 determines that the early data access request and the next data access request are the same request, the artificial neural network memory controller 120 can be configured to maintain the artificial neural network data locality pattern.

[0300] According to the above-described configuration, the artificial neural network memory system 100 detects that the artificial neural network model processed by the processor 110 is reused to prepare or provide data requested by the processor 110 faster.

[0301] When the artificial neural network memory controller 120 determines that the early data access request and the next data access request are different, the artificial neural network memory controller 120 can be configured to update the artificial neural network data locality pattern or further generate a new artificial neural network data locality pattern.

[0302] According to the above-described configuration, the artificial neural network memory system 100 can detect a change in the artificial neural network model processed by the processor 110 to generate an early data access request corresponding to the changed artificial neural network model.

[0303] In some example embodiments, the artificial neural network memory controller can be configured to generate a continuous early data access request.

[0304] For example, when the data access request token is [2], the early data access request generated by the artificial neural network memory controller can be a data access request corresponding to the token [3]. However, without being limited thereto, for example, the early data access request generated by the artificial neural network memory controller can be a plurality of data access requests corresponding to the tokens [3-4]. However, without being limited thereto, for example, the early data access request generated by the artificial neural network memory controller can be a plurality of data access requests corresponding to the tokens [3-4-5-6].

[0305] According to the above-described configuration, the artificial neural network memory controller can generate an early data access request predicting the entire order of the continuously repeated data access requests based on the artificial neural network data locality pattern.

[0306] According to the above-described configuration, the artificial neural network memory controller can generate an early data access request predicting the order of at least some of the data access requests based on the artificial neural network data locality pattern.

[0307] Figure 7 FIG. 7 is a flowchart schematically explaining an operation of an artificial neural network memory controller according to an example embodiment of the present disclosure.

[0308] Referring to Figure 7 For the artificial neural network operation processing, the processor 110 can be configured to generate data access requests corresponding to the artificial neural network model based on artificial neural network data locality.

[0309] The artificial neural network memory controller 120 sequentially records the data access requests generated in the processor 110 to generate an artificial neural network data locality pattern (S710).

[0310] The artificial neural network memory controller 120 compares the generated artificial neural network data locality pattern with the data access requests generated by the processor 110 to generate an advance data access request that predicts a next data access request to be generated by the processor 110.

[0311] The artificial neural network memory system 100 according to an example embodiment of the disclosure can be configured to include at least one processor 110 configured to generate data access requests corresponding to artificial neural network operations, and generate an artificial neural network data locality pattern of the artificial neural network operations by sequentially recording the data access requests (S720). The artificial neural network memory system 100 can be configured to include at least one artificial neural network memory controller 120 configured to generate an advance data access request that predicts a next data access request of the data access requests generated by the at least one processor 110 based on the artificial neural network data locality pattern.

[0312] That is, the at least one artificial neural network memory controller 120 generates the advance data access request before generating the next data access request (S730).

[0313] That is, the at least one processor 110 is configured to transmit the data access request to the at least one artificial neural network memory controller 120, and the at least one artificial neural network memory controller 120 can be configured to output the advance data access request corresponding to the data access request.

[0314] The artificial neural network memory system 100 according to an example embodiment of the disclosure can be configured to include at least one processor 110 and at least one artificial neural network memory controller 120, the at least one processor 110 being configured to generate data access requests corresponding to artificial neural network operations, and the at least one artificial neural network memory controller 120 being configured to generate an artificial neural network data locality pattern of the artificial neural network operations by sequentially recording the data access requests generated by the at least one processor 110, and generate an advance data access request that predicts a next data access request of the data access requests generated by the at least one processor 110 based on the artificial neural network data locality pattern.

[0315] According to the above-described configuration, the artificial neural network memory controller 120 predicts a next data access request pattern to be generated by an artificial neural network model being processed by the processor 110 based on artificial neural network data locality, so that advantageously, the corresponding data can be prepared in advance to be provided before a request of the processor 110.

[0316] The artificial neural network memory controller 120 can be configured to compare the generated early data access request with a next data access request generated by the processor 110 after the generation of the early data access request to determine an artificial neural network data locality pattern match (S740).

[0317] According to the above-described configuration, the artificial neural network memory controller 120 generates the early data access request before generating the next data access request in which data is prepared to be provided in advance. Accordingly, the artificial neural network memory controller 120 can substantially eliminate or reduce a delay that can occur when data is provided to the processor 110.

[0318] Figure 8 is a schematic block diagram for explaining an artificial neural network memory system according to another exemplary embodiment of the disclosure.

[0319] Referring to Figure 8 The artificial neural network memory system 200 can be configured to include a processor 210, an artificial neural network memory controller 220, and a memory 230.

[0320] When the artificial neural network memory system 200 according to another exemplary embodiment of the disclosure is compared with the artificial neural network memory system 100 according to one exemplary embodiment of the disclosure, the artificial neural network memory system 200 and the artificial neural network memory system 100 are substantially the same, except that the artificial neural network memory system 200 further includes the memory 230. Accordingly, redundant descriptions will be omitted for the convenience of description.

[0321] The artificial neural network memory system 200 according to another exemplary embodiment of the disclosure includes the memory 230 configured to communicate with the artificial neural network memory controller 220, and the memory 230 can be configured to operate according to a memory access request output from the artificial neural network memory controller 220.

[0322] The processor 210 can be configured to communicate with the artificial neural network memory controller 220. The processor 210 can be configured to generate a data access request to be transmitted to the artificial neural network memory controller 220. The data access request can be generated based on artificial neural network data locality of an artificial neural network model being processed. The processor 210 is configured to be provided with data corresponding to the data access request from the artificial neural network memory controller 220.

[0323] The artificial neural network memory controller 220 can be configured to receive a data access request generated by the processor 210. The artificial neural network memory controller 220 can be configured to generate an artificial neural network data locality pattern by analyzing artificial neural network data locality of an artificial neural network model being processed by the processor 210.

[0324] The artificial neural network memory controller 220 can be configured to control the memory 230 by generating a memory access request. The artificial neural network memory controller 220 can be configured to generate a memory access request corresponding to the data access request. That is, the artificial neural network memory controller 220 can be configured to generate a memory access request corresponding to the data access request generated by the processor 210. For example, when the artificial neural network memory controller 220 does not generate the artificial neural network data locality pattern, the artificial neural network memory controller 220 can be configured to generate a memory access request based on the data access request generated by the processor 210. In this case, the memory access request can be configured to include a memory address value and an operation mode value among the identification information included in the data access request.

[0325] The artificial neural network memory controller 220 can be configured to generate a memory access request corresponding to the advance data access request. That is, the artificial neural network memory controller 220 can be configured to generate a memory access request based on the advance data access request generated based on the artificial neural network data locality pattern. For example, when the artificial neural network memory controller 220 generates the artificial neural network data locality pattern, the artificial neural network memory controller 220 can be configured to generate a memory access request based on the advance data access request.

[0326] According to the above-described configuration, the artificial neural network memory controller 220 can transmit and receive data to and from the memory 230 through the memory access request, and when the memory access request is generated based on the advance data access request, the artificial neural network memory system 200 can provide data to the processor 210 more quickly.

[0327] The artificial neural network memory controller 220 can be configured to generate a memory access request based on one of the data access request generated by the processor 210 and the advance data access request generated by the artificial neural network memory controller 220. That is, the memory access request generated by the artificial neural network memory controller 220 can be selectively generated based on the data access request or the advance data access request.

[0328] The artificial neural network memory controller 220 can be configured to generate a memory access request including at least a part of the identification information included in the data access request and the early data access request. For example, the data access request generated by the processor 210 can include a memory address value and an operation mode value. At this time, the memory access request generated by the artificial neural network memory controller 220 can be configured to include the memory address value and the operation mode value of the corresponding data access request.

[0329] That is, each of the data access request, the early data access request, and the memory access request can be configured to include a corresponding memory address value and an operation mode value. The operation mode can be configured to include a read mode and a write mode. For example, the memory access request generated by the artificial neural network memory controller 220 can be configured to have the same configuration of data type as the data access request or the early data access request. Accordingly, from the perspective of the memory 230, the memory access request task can be performed according to the instruction of the artificial neural network memory controller 220 even without distinguishing the data access request and the early data access request.

[0330] According to the above-described configuration, the memory 230 can operate regardless of whether the memory access request generated by the artificial neural network memory controller 220 is based on the data access request or the early data access request. Accordingly, even if the artificial neural network memory controller 220 operates based on artificial neural network data locality, the artificial neural network memory controller can operate to be compatible with various types of memories.

[0331] The artificial neural network memory controller 220 transmits the memory access request to the memory 230, and the memory 230 performs a memory operation corresponding to the memory access request.

[0332] The memory according to an exemplary embodiment of the disclosure can be implemented in various forms. The memory can be implemented by a volatile memory and a non-volatile memory.

[0333] The volatile memory can include a dynamic RAM (DRAM) and a static RAM (SRAM). The non-volatile memory can include a programmable read-only memory (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a flash memory, a ferroelectric RAM (FRAM), a magnetic RAM (MRAM), and a phase change memory device (phase change RAM), but the disclosure is not limited thereto.

[0334] The memory 230 can be configured to store at least one of inference data, weight data, and feature map data of an artificial neural network model being processed by the processor 210. The inference data can be an input signal of the artificial neural network model.

[0335] The memory 230 can be configured to receive a memory access request from the artificial neural network memory controller 220. The memory 230 can be configured to perform a memory operation corresponding to the received memory access request. The operation mode of controlling the memory operation can include a read mode or a write mode.

[0336] For example, when the operation mode of the received memory access request is the write mode, the memory 230 can store the data received from the artificial neural network memory controller 220 in the corresponding memory address value.

[0337] For example, when the operation mode of the received memory access request is the read mode, the memory 230 can transfer the data stored in the corresponding memory address value to the artificial neural network memory controller 220. The artificial neural network memory controller 220 can be configured to transmit the received data to the processor 210 again.

[0338] The memory 230 can have a latency. The latency of the memory 230 can refer to a time in which the artificial neural network memory controller 220 is delayed in processing the memory access request. That is, when the memory 230 receives the memory access request from the artificial neural network memory controller 220, the actually requested data is output from the memory 230 after a delay of a certain clock period.

[0339] In order to process the memory access request, the memory 230 can access the memory address value included in the memory access request. Accordingly, a time to access the memory address value is necessary, and this time can be defined as a memory latency. For example, the CAS latency of a DDR4 SDRM memory is about 10 ns. When no data is provided to the processor 210 during the latency, the processor 210 is in an idle state, so that the processor cannot perform an actual operation.

[0340] In addition, in the case of a DRAM, which is one type of the memory 230, a plurality of clocks are used to activate a word line and a bit line according to a row address of the memory 230, a plurality of clocks are used to activate a column line, and a plurality of clocks are used to allow data to pass through a path through which data is transmitted outside the memory 230. Further, in the case of a NAND flash memory, a cell activated at one time is large, so that a plurality of clocks can be additionally used to search for data of a desired address therein.

[0341] The memory 230 can have a bandwidth. A data transfer rate of the memory 230 can be defined as a memory bandwidth. For example, a bandwidth of a DDR4 SDRAM memory is about 4 GByte / sec. As the memory bandwidth is higher, the memory 230 can transfer data to the processor 210 faster.

[0342] That is, a processing speed of the artificial neural network memory system 200 is affected by a latency generated when data to be processed by the processor 210 is provided and a bandwidth performance of the memory 230 more than a processing performance of the processor 210.

[0343] In other words, a bandwidth of the memory is gradually increasing, but an improvement of the latency of the memory is relatively slow compared to an improvement speed of the bandwidth. Specifically, a latency of the memory 230 is generated whenever a memory access request is generated, so that frequent memory access requests can be an important cause of a slow processing speed of the artificial neural network.

[0344] That is, even if an operation processing speed of the processor 210 is fast, if a latency is generated to acquire data required for an operation, the processor 210 can be in an idle state in which the operation is not performed. Accordingly, in this case, the operation processing speed of the processor 210 can be slowed down.

[0345] Accordingly, the artificial neural network memory system according to an example embodiment of the disclosure can be configured to improve a bandwidth and / or a latency of the memory 230.

[0346] Figure 9 is a schematic diagram for explaining an operation of a memory system according to a comparative embodiment of the disclosure.

[0347] Referring to Figure 9 , the processor generates a data access request, and the memory system of the related art can transmit a memory access request corresponding to the data access request to the memory. At this time, the memory has a latency so that the requested data can be provided from the memory to the processor after waiting for the latency.

[0348] For example, the memory system of the related art receives a data access request [1] generated by the processor and transmits a memory access request [1'] corresponding to the data access request [1] to the memory. The memory can transmit data [1''] to the memory system after a latency. Accordingly, at each data access request, a processing time of the processor can be delayed as much as the latency of the memory. Accordingly, a time of an inference operation of the artificial neural network can be as much as the latency of the memory. Specifically, as the processor generates more data access requests, the artificial neural network inference operation time of the memory system of the related art can be further delayed.

[0349] Figure 10 is a schematic diagram for explaining a memory system according to another exemplary embodiment of the disclosure.

[0350] Referring to Figure 10 , the processor 210 generates a data access request [1], and the artificial neural network memory controller 220 can transmit a memory access request corresponding to an advance data access request generated based on an artificial neural network data locality pattern to the memory 230. At this time, even if the memory 230 has a delay, the processor 210 will generate a memory access request corresponding to the advance data access request, so that when the processor 210 generates a next data access request, the artificial neural network memory controller 220 can directly provide data requested by the processor 210 to the processor 210.

[0351] For example, the data access request [1] generated by the processor 210 is received by the artificial neural network memory controller 220 to generate an advance data access request [2] and transmit a memory access request [2'] corresponding to the advance data access request [2] to the memory 230. The memory 230 can transmit data [2"] to the artificial neural network memory controller 220 after a delay. However, the data [2"] provided by the memory 230 is data corresponding to the memory access request [2'] based on the advance data access request [2]. Therefore, when the processor 210 generates a next data access request [2], the artificial neural network memory controller 220 can immediately provide the data [2"] to the processor 210.

[0352] If the time between the memory access request based on the advance data access request and the next data access request is longer than the delay of the memory 230, the artificial neural network memory controller 220 can provide data to the processor 210 as soon as the next data access request is received from the processor 210. In this case, the artificial neural network memory controller 220 can substantially eliminate the delay of the memory 230.

[0353] In other words, when the memory access request based on the advance data access request is transmitted to the memory 230, the delay of the memory 230 can be shorter than or equal to the time from the generation of the advance data access request to the generation of the next data access request. In this case, the artificial neural network memory controller 220 can immediately provide data without causing a delay as soon as the processor 210 generates the next data access request.

[0354] Even if the time between the memory access request based on the advance data access request and the next data access request is shorter than the delay of the memory 230, the delay of the memory 230 is significantly reduced as much as the time between the memory access request and the next data access request.

[0355] According to the above-described configuration, the artificial neural network memory controller 220 can substantially eliminate or reduce a latency of data to be provided to the processor 210.

[0356] In some example embodiments, the artificial neural network memory controller of the artificial neural network memory system can be configured to measure a latency of the memory or be provided with a latency value of the memory from the memory.

[0357] According to the above-described configuration, the artificial neural network memory controller can be configured to determine a timing of generating a memory access request based on an early data access request based on a latency of the memory. Accordingly, the artificial neural network memory controller can generate a memory access request based on an early data access request that substantially minimizes the latency of the memory.

[0358] In some example embodiments, the memory of the artificial neural network memory system can be a memory configured to include a refresh function of updating a voltage of a memory cell. The artificial neural network memory controller can be configured to selectively control a refresh of a memory address region of the memory corresponding to a memory access request corresponding to an early data access request. For example, the memory can be a DRAM including a refresh function.

[0359] If the DRAM does not refresh the voltage of the memory cell, the memory cell slowly discharges such that the stored data can be lost. Accordingly, the voltage of the memory cell needs to be refreshed at every certain period. If the timing of the memory access request of the artificial neural network memory controller overlaps with the refresh timing, the artificial neural network memory system can be configured to advance or delay the timing of refreshing the voltage of the memory cell.

[0360] The artificial neural network memory system can predict or calculate the timing of generating the memory access request based on the artificial neural network data locality pattern. Accordingly, the artificial neural network memory system can be configured to limit the voltage refresh of the memory cell during the memory access request operation.

[0361] In other words, the inference operation of the artificial neural network operation operates with a concept of accuracy such that even if the stored data is partially lost due to the delayed refresh of the voltage of the memory cell, a decrease in inference accuracy can be substantially negligible.

[0362] According to the above-described configuration, data can be provided to the artificial neural network memory system according to the memory access request from the memory by adjusting the refresh period of the voltage of the memory cell. Accordingly, it is possible to improve the operation speed of the artificial neural network that is decreased according to the refresh of the voltage of the memory cell without significantly decreasing the inference accuracy.

[0363] In some example embodiments, the memory of the artificial neural network memory system can be configured to further include a pre-charge function that charges global bit lines of the memory with a specific voltage. At this time, the artificial neural network memory controller can be configured to selectively provide the pre-charge to a memory address region of the memory corresponding to a memory access request corresponding to an early data access request.

[0364] In some example embodiments, the artificial neural network memory controller can be configured to pre-charge or delay a bit line of the memory that performs a memory task corresponding to an early data access request based on a pattern of artificial neural network data locality.

[0365] Generally, the memory performs a pre-charge operation by receiving a memory access request to perform a read operation or a write operation. When one memory operation is completed, a signal remains in a bit line that performs a data read / write operation and each data input / output line, so that a subsequent memory operation can be smoothly performed only when the above lines are pre-charged to a predetermined level. However, since the time required for pre-charging is quite long, when the timing of generating a memory access request overlaps with the timing of pre-charging, the memory operation can be delayed by the pre-charging time. Accordingly, the time of processing a data access request requested by a processor is delayed.

[0366] The artificial neural network memory controller can predict a memory operation to be performed on a bit line of a specific memory in a specific order based on a pattern of artificial neural network data locality. Accordingly, the artificial neural network memory controller can pre-charge or delay a pre-charge time so that the pre-charge timing and the time of performing a memory operation on a specific bit line do not overlap.

[0367] In other words, an inference operation of an artificial neural network model operates with the concept of accuracy, so that even if the stored data is partially lost due to delayed pre-charging, the reduction in inference accuracy is substantially negligible.

[0368] In other words, an artificial neural network is a mathematical model that is modeled by simulating a brain neural network of a biological system. Human nerve cells called neurons exchange information through junctions called synapses between nerve cells, and the exchange of information between nerve cells is very simple, but a large number of nerve cells are gathered to create intelligence. The advantage of this structure is that even if some nerve cells transmit incorrect information, it does not affect the overall information, so that it has strong robustness to small errors. Accordingly, due to the above-described characteristics, even if the pre-charging and refresh functions of the memory storing the data of the artificial neural network model are selectively restricted, the accuracy of the artificial neural network model is substantially not problematic and the memory delay due to pre-charging or refresh is reduced.

[0369] According to the above-described configuration, it is possible to improve the operation speed reduction according to the pre-charged artificial neural network without substantially reducing the inference accuracy.

[0370] In some example embodiments, the artificial neural network memory controller can be configured to independently control the refresh function and the pre-charge function of the memory based on the artificial neural network data locality pattern.

[0371] Figure 11 is a schematic block diagram for explaining an artificial neural network memory system according to still another example embodiment of the present disclosure.

[0372] Referring to Figure 11 The artificial neural network memory system 300 can be configured to include a processor 310, an artificial neural network memory controller 320 including a cache memory 322, and a memory 330.

[0373] When comparing the artificial neural network memory system 300 according to another example embodiment of the present disclosure with the artificial neural network memory system 200 according to another example embodiment of the present disclosure, the artificial neural network memory system 300 and the artificial neural network memory system 200 are substantially the same, except that the artificial neural network memory system 300 further includes the cache memory 322. Thus, for the convenience of description, redundant descriptions will be omitted.

[0374] The artificial neural network memory system 300 according to still another example embodiment of the present disclosure can be configured to include an artificial neural network memory controller 320 including a cache memory 322 configured to store data transmitted by the memory 330 in response to a memory access request based on an advance data access request.

[0375] According to the above-described configuration, the artificial neural network memory controller 320 can read data in response to a memory access request based on an advance data access request from the memory 330 and store the data in the cache memory 322. Thus, when the processor 310 generates a next data access request, the artificial neural network memory controller 320 can immediately provide the data stored in the cache memory 322 to the processor 310.

[0376] The latency of the cache memory 322 is much shorter than that of the memory 330. The bandwidth of the cache memory 322 is higher than that of the memory 330.

[0377] The artificial neural network model processing performance of the artificial neural network memory system 300 including the cache 322 according to yet another exemplary embodiment of the disclosure can be superior to that of the artificial neural network memory system 200 according to another exemplary embodiment of the disclosure.

[0378] An artificial neural network model 1300 will be described with reference to FIG. 13, according to yet another exemplary embodiment of the disclosure. Figure 3 The artificial neural network model 1300 can be compiled by a specific compiler to operate in the processor 310. The compiler can be configured to provide the artificial neural network data locality pattern to the artificial neural network memory controller 320.

[0379] To infer the artificial neural network model 1300, the processor 310 can be configured to generate data access requests according to an order based on the artificial neural network data locality. Accordingly, the artificial neural network memory controller 320 can monitor the data access requests to generate the artificial neural network data locality pattern 1400. Alternatively, the artificial neural network memory controller 320 can store the artificial neural network data locality pattern 1400 that has been generated in advance.

[0380] Hereinafter, an example in which the artificial neural network data locality pattern 1400 is not generated will be described.

[0381] First, the processor 310 can generate a data access request of a token [1] corresponding to a node value read pattern of the input layer 1310. Accordingly, the artificial neural network memory controller 320 generates a memory access request of the token [1] to transmit the node value of the input layer 1310 transmitted from the memory 330 to the processor 310.

[0382] Next, the processor 310 can generate a data access request of a token [2] corresponding to a weight value of the first connection network 1320. Accordingly, the artificial neural network memory controller 320 generates a memory access request of the token [2] to transmit the weight value of the first connection network 1320 transmitted from the memory 330 to the processor 310.

[0383] Next, the processor 310 receives the node value of the input layer 1310 and the weight value of the first connection network 1320 to calculate a node value of the first hidden layer 1330. That is, the processor 310 can generate a data access request of a token [3] corresponding to a node value write pattern of the first hidden layer 1330. Accordingly, the artificial neural network memory controller 320 generates a memory access request of the token [3] to store the node value of the first hidden layer 1330 in the memory 330.

[0384] Next, the processor 310 receives the node value of the input layer 1310 and the weight value of the first connection network 1320 to calculate a node value of the first hidden layer 1330. That is, the processor 310 can generate a data access request of a token [3] corresponding to a node value write pattern of the first hidden layer 1330. Accordingly, the artificial neural network memory controller 320 generates a memory access request of the token [3] to store the node value of the first hidden layer 1330 in the memory 330.

[0385] Next, the processor 310 can generate a data access request of a token [4] corresponding to the node value read pattern of the first hidden layer 1330. Accordingly, the artificial neural network memory controller 320 generates a memory access request of the token [4] to transmit the node value of the first hidden layer 1330 transmitted from the memory 330 to the processor 310.

[0386] Next, the processor 310 can generate a data access request of a token [5] corresponding to the weight value of the second connection network 1340. Accordingly, the artificial neural network memory controller 320 generates a memory access request of the token [5] to transmit the weight value of the second connection network 1340 transmitted from the memory 330 to the processor 310.

[0387] Next, the processor 310 receives the node value of the first hidden layer 1330 and the weight value of the second connection network 1340 to calculate the node value of the second hidden layer 1350. That is, the processor 310 can generate a data access request of a token [6] corresponding to the node value write pattern of the second hidden layer 1350. Accordingly, the artificial neural network memory controller 320 generates a memory access request of the token [6] to store the node value of the second hidden layer 1350 in the memory 330.

[0388] Next, the processor 310 can generate a data access request of a token [7] corresponding to the node value read pattern of the second hidden layer 1350. Accordingly, the artificial neural network memory controller 320 generates a memory access request of the token [7] to transmit the node value of the second hidden layer 1350 transmitted from the memory 330 to the processor 310.

[0389] Next, the processor 310 can generate a data access request of a token [8] corresponding to the weight value of the third connection network 1360. Accordingly, the artificial neural network memory controller 320 generates a memory access request of the token [8] to transmit the weight value of the third connection network 1360 transmitted from the memory 330 to the processor 310.

[0390] Next, the processor 310 receives the node value of the second hidden layer 1350 and the weight value of the third connection network 1360 to calculate the node value of the output layer 1370. That is, the processor 310 can generate a data access request of a token [9] corresponding to the node value write pattern of the output layer 1370. Accordingly, the artificial neural network memory controller 320 generates a memory access request of the token [9] to store the node value of the output layer 1370 in the memory 330.

[0391] Accordingly, the artificial neural network memory system 300 can store the inference result of the artificial neural network model 1300 in the output layer 1370.

[0392] In the above example, the artificial neural network data locality pattern 1400 has not been generated in the artificial neural network memory controller 320. Accordingly, according to the above example, the early data access request cannot be generated. Accordingly, since the artificial neural network memory controller 320 does not provide data in advance, a delay of the memory 330 can be caused in each memory access request.

[0393] However, since the artificial neural network memory controller 320 records the data access request, when the processor 310 generates the data access request of the token [1] corresponding to the node value read pattern of the input layer 1310 again, the artificial neural network data locality pattern 1400 can be generated.

[0394] Hereinafter, the reference Figure 4 The artificial neural network data locality pattern 1400 is generated.

[0395] In the following example, the artificial neural network data locality pattern 1400 is generated and the processor 310 repeatedly infers the artificial neural network model 1300, but is not limited thereto.

[0396] The processor 310 detects the repeated data access request of the token [1] to generate the artificial neural network data locality pattern 1400. In other words, since the artificial neural network memory controller 320 sequentially stores from the token [1] to the token [9], when the artificial neural network memory controller 320 detects the token [1] again, the artificial neural network data locality can be determined.

[0397] However, as described above, the artificial neural network memory controller according to an example embodiment of the disclosure is not limited to the token. The token is used for convenience of description only, and an example embodiment of the disclosure can be implemented by identification information included in the data access request and the memory access request.

[0398] For example, when the processor 310 generates the data access request corresponding to the token [9], the artificial neural network memory controller 320 generates the early data access request of the token [1]. Accordingly, the artificial neural network memory controller 320 generates the memory access request of the token [1] to store the node value of the input layer 1310 in the cache 322 in advance.

[0399] That is, if the data access request of the token [9] is the last step of the artificial neural network model 1300, the artificial neural network memory controller 320 can predict that the data access request of the token [1] which is the start step of the artificial neural network model 1300 will be generated.

[0400] Next, when the processor 310 generates the data access request of the token [1], the artificial neural network memory controller 320 determines whether the early data access request of the token [1] and the data access request of the token [1] are the same. When it is determined that the requests are the same, the node value of the input layer 1310 stored in the cache memory 322 can be immediately provided to the processor 310.

[0401] At this time, the artificial neural network memory controller 320 generates the early data access request of the token [2].

[0402] Accordingly, the artificial neural network memory controller 320 generates the memory access request of the token [2] to store the weight value of the first connection network 1320 in the cache memory 322 in advance.

[0403] Next, when the processor 310 generates the data access request of the token [2], the artificial neural network memory controller 320 determines whether the early data access request of the token [2] and the data access request of the token [2] are the same. When it is determined that the requests are the same, the node value of the first connection network 1320 stored in the cache memory 322 can be immediately provided to the processor 310.

[0404] At this time, the artificial neural network memory controller 320 generates the early data access request of the token [3].

[0405] Next, the processor 310 receives the node value of the input layer 1310 and the weight value of the first connection network 1320 to calculate the node value of the first hidden layer 1330. When the processor 310 generates the data access request of the token [3], the artificial neural network memory controller 320 determines whether the early data access request of the token [3] and the data access request of the token [3] are the same. When it is determined that the requests are the same, the calculated node value of the first hidden layer 1330 can be stored in the memory 330 and / or the cache memory 322.

[0406] The cache memory 322 will be described additionally. When the same data is stored in the memory 330 as the memory access request of the token [3] without the cache memory 322 and then read from the memory 330 as the memory access request of the token [4], the delay of the memory 330 can be doubled.

[0407] In this case, the artificial neural network memory controller 320 stores the node value of the layer calculated based on the fact that the memory address values of the consecutive tokens are the same and the operation mode of the previous token is the write mode, the operation mode of the subsequent token is the read mode, and determines to use the corresponding node value as the input value of the subsequent layer.

[0408] That is, when the data of the token [3] is stored in the cache memory 322, the data access requests corresponding to the token [3] and the token [4] can be processed in the cache memory 322. Accordingly, the artificial neural network memory controller 320 can be configured not to generate the memory access request corresponding to the data access request of the token [3] and the data access request of the token [4]. According to the above-described configuration, the latency of the memory 330 to the memory 330 can be eliminated by the memory access request of the token [3] and the memory access request of the token [4]. In particular, the cache memory 322 operation strategy can be performed based on the artificial neural network data locality pattern 1400.

[0409] At this time, the artificial neural network memory controller 320 generates the early data access request of the token [4].

[0410] Next, when the processor 310 generates the data access request of the token [4], the artificial neural network memory controller 320 determines whether the early data access request of the token [4] and the data access request of the token [4] are the same. When it is determined that the requests are the same, the node value of the first hidden layer 1330 stored in the cache memory 322 can be immediately provided to the processor 310.

[0411] At this time, the artificial neural network memory controller 320 generates the early data access request of the token [5].

[0412] Accordingly, the artificial neural network memory controller 320 generates the memory access request of the token [5] to store the weight value of the second connection network 1340 in the cache memory 322 in advance.

[0413] Next, when the processor 310 generates the data access request of the token [5], the artificial neural network memory controller 320 determines whether the early data access request of the token [5] and the data access request of the token [5] are the same. When it is determined that the requests are the same, the weight value of the second connection network 1340 stored in the cache memory 322 can be immediately provided to the processor 310.

[0414] At this time, the artificial neural network memory controller 320 generates the early data access request of the token [6].

[0415] Next, the processor 310 receives the node values of the first hidden layer 1330 and the weight values of the second connection network 1340 to compute the node values of the second hidden layer 1350. When the processor 310 generates the data access request for token [6], the artificial neural network memory controller 320 determines whether the early data access request for token [6] and the data access request for token [6] are the same. When the requests are determined to be the same, the computed node values of the second hidden layer 1350 can be stored in the memory 330 and / or the cache memory 322.

[0416] At this time, the artificial neural network memory controller 320 generates the early data access request for token [7].

[0417] Next, when the processor 310 generates the data access request for token [7], the artificial neural network memory controller 320 determines whether the early data access request for token [7] and the data access request for token [7] are the same. When the requests are determined to be the same, the node values of the second hidden layer 1350 stored in the cache memory 322 can be immediately provided to the processor 310.

[0418] At this time, the artificial neural network memory controller 320 generates the early data access request for token [8].

[0419] Accordingly, the artificial neural network memory controller 320 generates the memory access request for token [8] to early store the weight values of the third connection network 1360 in the cache memory 322.

[0420] Next, when the processor 310 generates the data access request for token [8], the artificial neural network memory controller 320 determines whether the early data access request for token [8] and the data access request for token [8] are the same. When the requests are determined to be the same, the weight values of the third connection network 1360 stored in the cache memory 322 can be immediately provided to the processor 310.

[0421] At this time, the artificial neural network memory controller 320 generates the early data access request for token [9].

[0422] Next, the processor 310 receives the node values of the second hidden layer 1350 and the weight values of the third connection network 1360 to compute the node values of the output layer 1370. When the processor 310 generates the data access request for token [9], the artificial neural network memory controller 320 determines whether the early data access request for token [9] and the data access request for token [9] are the same. When the requests are determined to be the same, the computed node values of the output layer 1370 can be stored in the memory 330 and / or the cache memory 322.

[0423] Accordingly, the artificial neural network memory system 300 can store inference results of the artificial neural network model 1300 in the output layer 1370.

[0424] Even if the inference of the artificial neural network model 1300 ends in the artificial neural network data locality pattern 1400, the artificial neural network memory system 300 can be ready to immediately start the next inference.

[0425] That is, the artificial neural network memory system 300 according to still another exemplary embodiment of the disclosure can be configured to generate an early data access request based on artificial neural network data locality, determine whether the early data access request is identical to an actual data access request, and if the requests are identical, further generate a next early data access request. According to the above configuration, the artificial neural network memory controller 320 can eliminate or reduce the latency of the memory 330 in processing the data access request.

[0426] In some exemplary embodiments, the artificial neural network memory controller can be configured to operate to minimize the available space of the cache memory by generating at least one early data access request.

[0427] That is, the artificial neural network memory controller compares the memory available space of the cache memory and the size of the data value to be stored, and when the memory available space of the cache memory exists, generates at least one early data access request to minimize the available space of the cache memory.

[0428] That is, the artificial neural network memory controller can be configured to generate a plurality of early data access requests according to the capacity of the cache memory.

[0429] That is, the artificial neural network memory controller can be configured to sequentially generate at least one memory access request to minimize the remaining capacity of the cache memory based on the remaining capacity of the cache memory.

[0430] Reference will be made to Figures 2 to 6The example is described. When the processor generates a data access request for token [1], the artificial neural network memory controller generates an early data access request for token [2] to pre-store the weight values of the first connection network 1320 in the cache memory. Next, the artificial neural network memory controller can pre-allocate the space for storing and reading the node value calculation results of the first hidden layer 1330 corresponding to tokens [3] and [4] to the cache memory. Next, the artificial neural network memory controller can pre-store the weight values of the second connection network 1340 corresponding to token [5] in the cache memory. When there is a margin in the capacity of the cache memory, the artificial neural network memory controller can be configured to further generate early data access requests in sequence according to the artificial neural network data locality pattern. That is, when there is a margin in the capacity of the cache memory, the artificial neural network memory controller can be configured to pre-store the weight values in the cache memory or to pre-ensure the area for storing the artificial neural network operation results according to the artificial neural network data locality pattern.

[0431] If the capacity of the cache memory is sufficient, the weight values of all connection networks of the artificial neural network model 1300 can be stored in the cache memory. Specifically, in the case of a learned artificial neural network model, the weight values are fixed. Therefore, when the weight values reside in the cache memory, the memory delay caused by reading the weight values through the memory access request can be eliminated.

[0432] According to the above configuration, the data required for the cache memory to store based on the artificial neural network data locality is optimized to improve the operating efficiency of the cache memory and to improve the processing speed of the artificial neural network memory system 300.

[0433] According to the above configuration, considering both the artificial neural network data locality pattern and the capacity of the cache memory, the cache memory sequentially generates early data access requests, so that the processing speed of the artificial neural network memory system can be improved.

[0434] According to the above configuration, when the processor generates a specific data access request included in the artificial neural network data locality pattern 1400, the artificial neural network memory controller can sequentially predict at least one data access request after the specific data access request. For example, when the processor generates a data access request for token [1], the artificial neural network memory controller can predict that the corresponding data access requests are generated in the order of tokens [2-3-4-5-6-7-8-9].

[0435] According to the above-described configuration, the artificial neural network memory controller 320 can cause a certain weight value to reside in the cache memory for a certain period of time. For example, when the processor performs inference at a speed of 30 times per second by utilizing the artificial neural network model, the weight value of a certain layer can reside in the cache memory. In this case, the artificial neural network memory controller can reuse the weight value stored in the cache memory for each inference. Accordingly, the corresponding memory access request can be selectively deleted. Thus, a delay according to the memory access request can be eliminated.

[0436] In some example embodiments, the cache memory can be configured by a plurality of hierarchical cache memories. For example, the cache memory can include a cache memory configured to store a weight value or a cache memory configured to store a feature map.

[0437] In some example embodiments, when the artificial neural network data locality pattern 1400 is generated, the artificial neural network memory controller can be configured to predict a weight value and a node value based on identification information included in the data access request. Accordingly, the artificial neural network memory controller can be configured to identify a data access request corresponding to a weight value. Specifically, when it is assumed that completion of learning causes a weight value of a connected network to be fixed, in the artificial neural network data locality pattern 1400, the weight value can be configured to operate only in a read mode. Accordingly, the artificial neural network memory controller can determine tokens [2], [5], and [8] as weight values. In other words, token [1] is a starting step of inference, so it can be determined as an input node value. In other words, token [9] is a last step of inference, so it can be determined as an output node value. In other words, tokens [3] and [4] have a sequence of a write mode and a read mode of the same memory address value, so tokens [3] and [4] can be determined as node values of a hidden layer. However, it can vary according to artificial neural network data locality of the artificial neural network model.

[0438] The artificial neural network memory controller can be configured to analyze the artificial neural network data locality pattern to determine whether a data access request is a weight value, a kernel window value, a node value, an activation map value, etc. of the artificial neural network model.

[0439] In some example embodiments, an artificial neural network memory system includes a processor configured to generate a data access request corresponding to an artificial neural network operation, an artificial neural network memory controller configured to store an artificial neural network data locality pattern generated by a compiler and to generate an advance data access request that is a next data access request predicted by the processor generated data access request based on the artificial neural network data locality pattern, and a memory configured to communicate with the artificial neural network memory controller. The memory can be configured to operate according to a memory access request output from the artificial neural network memory controller.

[0440] According to the above-described configuration, the artificial neural network memory controller can be configured to have the artificial neural network data locality pattern generated from the compiler. In this case, the artificial neural network memory controller can allow the data access request of the artificial neural network model being processed by the processor to be prepared in advance in the cache memory based on the artificial neural network data locality pattern generated by the compiler. Specifically, the artificial neural network data locality pattern generated by the compiler can be more accurate than the artificial neural network data locality pattern generated by monitoring the artificial neural network data locality.

[0441] In other words, the artificial neural network memory controller can be configured to store the artificial neural network data locality pattern generated by the compiler and the artificial neural network data locality pattern generated by independently monitoring the data access request, respectively.

[0442] Figure 12 is a schematic diagram for illustrating example identification information of a data access request.

[0443] The data access request generated by the processor according to the example embodiments of the present disclosure can be configured to further include at least one additional identification information. The additional identification information can also be referred to as a sideband signal or sideband information.

[0444] The data access request generated by the processor can be an interface signal having a specific structure. That is, the data access request can be an interface signal for the processor and the artificial neural network memory controller to communicate. The data access request can be configured to further include an additional bit to additionally provide identification information required for the artificial neural network operation, but the present disclosure is not limited thereto, and the additional identification information can be provided in various ways.

[0445] In some example embodiments, the data access request of the artificial neural network memory system can be configured to further include identification information to identify whether it is an artificial neural network operation, but the example embodiments of the present disclosure are not limited thereto.

[0446] For example, the artificial neural network memory system adds a one-bit identification code in the data access request to identify whether the data access request received by the artificial neural network memory controller is a data access request related to the artificial neural network operation. However, the number of bits of the identification code according to the example embodiments of the disclosure is not limited and can be adjusted according to the number of cases of objects to be identified.

[0447] For example, when the identification code is [0], the artificial neural network memory controller can determine that the corresponding data access request is related to the artificial neural network operation.

[0448] For example, when the identification code is [1], the artificial neural network memory controller can determine that the corresponding data access request is not related to the artificial neural network operation.

[0449] In this case, the artificial neural network memory controller can be configured to generate an artificial neural network data locality pattern by recording only the data access request related to the artificial neural network operation based on the identification information included in the data access request. According to the above-described configuration, the artificial neural network memory controller can not record the data access request not related to the artificial neural network operation. By doing so, the accuracy of the artificial neural network data locality pattern generated by recording the data access request can be improved, but the example embodiments of the disclosure are not limited thereto.

[0450] In some example embodiments, the data access request of the artificial neural network memory system can be configured to further include identification information to identify whether the artificial neural network operation is an operation for learning or an operation for inference, but the example embodiments of the disclosure are not limited thereto.

[0451] For example, the artificial neural network memory system adds a one-bit identification code in the data access request so that the data access request received through the artificial neural network memory controller is configured to identify whether the operation type of the artificial neural network model is learning or inference. However, the number of bits of the identification code according to the example embodiments of the disclosure is not limited and can be adjusted according to the number of cases of objects to be identified.

[0452] For example, when the identification code is [0], the artificial neural network memory controller can determine that the corresponding data access request is a learning operation.

[0453] For example, when the identification code is [1], the artificial neural network memory controller can determine that the corresponding data access request is an inference operation.

[0454] In this case, the artificial neural network memory controller can be configured to generate the artificial neural network data locality pattern by separately recording the data access requests of the learning operation and the data access requests of the inference operation. For example, in the learning mode, it is also possible to include an evaluation step of determining the inference accuracy of the trained artificial neural network model, and updating the weight values of each layer and / or kernel window of the artificial neural network model. Accordingly, even if the structure of the artificial neural network model is the same, the artificial neural network data locality to be processed by the processor can be different in the learning operation and the inference operation.

[0455] According to the above-described configuration, the artificial neural network memory controller can be configured to generate the artificial neural network data locality pattern of the learning mode and the artificial neural network data locality pattern of the inference mode for a specific artificial neural network model, respectively. By doing so, it is possible to improve the accuracy of the artificial neural network data locality pattern generated by the artificial neural network memory controller by recording the data access requests, but exemplary embodiments of the present disclosure are not limited thereto.

[0456] In some exemplary embodiments, the data access request of the artificial neural network memory system can be configured with an operation mode including identification information to identify the memory read operation and the memory write operation, but is not limited thereto, such that the data access request of the artificial neural network memory system can be configured with an operation mode further including identification information for identifying the overwrite operation and / or the protection operation, but exemplary embodiments of the present disclosure are not limited thereto.

[0457] For example, one bit of identification code is added to the data access request of the artificial neural network memory system to include the read operation and the write operation. Alternatively, two bits of identification code are added to the data access request of the artificial intelligence network storage system to identify the read operation, the write operation, the overwrite, and the protection operation. However, the number of bits of the identification code according to exemplary embodiments of the present disclosure is not limited and can be adjusted according to the number of cases of objects to be identified.

[0458] In other words, for the operation of the artificial neural network memory system, the data access request needs to include identification information to identify the memory address value and the read operation, and the write operation. The artificial neural network memory controller receives the data access request to generate a corresponding memory access request to perform the memory operation.

[0459] For example, when the identification code is

[00] , the artificial neural network memory controller can determine the corresponding data access request as the read operation.

[0460] For example, when the identification code is

[01] , the artificial neural network memory controller can determine the corresponding data access request as the write operation.

[0461] For example, when the identification code is

[10] , the artificial neural network memory controller can determine the corresponding data access request as an overwrite operation.

[0462] For example, when the identification code is

[11] , the artificial neural network memory controller can determine the corresponding data access request as a protection operation.

[0463] However, exemplary embodiments of the disclosure are not limited thereto.

[0464] According to the above-described configuration, the artificial neural network memory controller controls the memory according to the read mode or the write mode to provide or store various data of the artificial neural network model in the memory.

[0465] According to the above-described configuration, the artificial neural network memory controller can update the weight values of a specific layer through an overwrite operation during a learning operation of the artificial neural network. Specifically, the updated weight values are stored in the same memory address value, so that a new memory address can not be allocated. Accordingly, the overwrite mode can be more efficient than the write mode during the learning operation.

[0466] According to the above-described configuration, the artificial neural network memory controller can protect the data stored in a specific memory address through a protection mode. Specifically, in an environment where a plurality of users simultaneously access, such as an ae server, the data of the artificial neural network model cannot be arbitrarily eliminated. In addition, the weight values of the artificial neural network model that ends learning can be protected through the protection mode.

[0467] In some exemplary embodiments, the data access request of the artificial neural network memory system can be configured to further include identification information capable of identifying inference data, weights, feature maps, learning data sets, evaluation data sets, and the like, but exemplary embodiments of the disclosure are not limited thereto.

[0468] For example, the artificial neural network memory system can be configured to add a 3-bit identification code to the data access request so that the artificial neural network memory controller can identify the domain of the data to be accessed. However, the number of bits of the identification code according to exemplary embodiments of the disclosure is not limited and can be adjusted according to the number of cases of objects to be identified.

[0469] For example, when the identification code is

[000] , the artificial neural network memory controller can determine that the corresponding data is data irrelevant to the artificial neural network model.

[0470] For example, when the identification code is

[001] , the artificial neural network memory controller can determine that the corresponding data is inference data of the artificial neural network model.

[0471] For example, when the identification code is

[010] , the artificial neural network memory controller can determine that the corresponding data is a feature map of the artificial neural network model.

[0472] For example, when the identification code is

[011] , the artificial neural network memory controller can determine that the corresponding data is a weight of the artificial neural network model.

[0473] For example, when the identification code is

[100] , the artificial neural network memory controller can determine that the corresponding data is a learning data set of the artificial neural network model.

[0474] For example, when the identification code is

[101] , the artificial neural network memory controller can determine that the corresponding data is an inference data set of the artificial neural network model.

[0475] According to the above-described configuration, the artificial neural network memory controller can be configured to identify a domain of data of the artificial neural network model and allocate an address of a memory storing data corresponding to the domain. For example, the artificial neural network memory controller can set a start address and an end address of a memory region allocated to the domain. According to the above-described configuration, data allocated to the domain can be stored in an order corresponding to the artificial neural network data locality pattern.

[0476] For example, data of a domain of the artificial neural network model can be sequentially stored in a memory region allocated to the domain. At this time, the memory can be a memory supporting a read burst function. According to the above-described configuration, when the artificial neural network memory controller reads data of a specific domain from the memory, the specific data can be configured to be stored to be optimized for the read burst function according to the artificial neural network data locality pattern. That is, the artificial neural network memory controller can be configured to set a storage region of the memory in consideration of the read burst function.

[0477] In some example embodiments, the memory further includes a read burst function, and at least one artificial neural network memory controller can be configured to write a storage region of at least one memory in consideration of the read burst function.

[0478] In some example embodiments, the data access request of the artificial neural network memory system can be configured to further include identification information to identify quantization of the artificial neural network model, but example embodiments of the disclosure are not limited thereto.

[0479] For example, when the data access request includes at least a memory address value, a domain, and quantization identification information, the artificial neural network memory system can be configured to identify quantization information of data of the domain.

[0480] For example, when the identification code is

[00001] , the artificial neural network memory controller can determine that the corresponding data is data quantized to one bit.

[0481] For example, when the identification code is

[11111] , the artificial neural network memory controller can determine that the corresponding data is data quantized to 32 bits.

[0482] In some exemplary embodiments, various identification information can be selectively included in the data access request.

[0483] According to the above configuration, the artificial neural network memory controller analyzes the identification code of the data access request to generate more accurate artificial neural network data locality patterns. Further, each identification information is calculated to selectively control the storage strategy of the memory.

[0484] For example, when identifying learning and inference, each artificial neural network data locality pattern can be generated.

[0485] For example, when the domain of the data is identified, a strategy to store the data of the artificial neural network data locality pattern in a specific memory area is established to improve the efficiency of memory operations.

[0486] In some exemplary embodiments, when the artificial neural network memory system is configured to process multiple artificial neural network models, the artificial neural network memory controller can be configured to further generate identification information of the artificial neural network model, for example, additional identification information, for example, a first artificial neural network model or a second artificial neural network model. At this time, the artificial neural network memory controller can be configured to distinguish the artificial neural network model based on the artificial neural network data locality of the artificial neural network model, but is not limited thereto.

[0487] Figure 13 is a schematic diagram for explaining the energy consumption per unit operation of the artificial neural network memory system.

[0488] Referring to Figure 13 In the table, the energy consumed per unit operation of the artificial neural network memory system 300 is schematically explained. The energy consumption can be explained as allocated to memory access, addition operation, and multiplication operation.

[0489] "8b Add" refers to an 8-bit integer addition operation of an adder. The 8-bit integer addition operation can consume 0.03 pj of energy.

[0490] "16b Add" refers to a 16-bit integer addition operation of an adder. The 16-bit integer addition operation can consume 0.05 pj of energy.

[0491] “32b Add” refers to a 32-bit integer addition operation of an adder. A 32-bit integer addition operation can consume 0.1 pj of energy.

[0492] “16b FP Add” refers to a 16-bit floating point addition operation of an adder. A 16-bit floating point addition operation can consume 0.4 pj of energy.

[0493] “32b FP Add” refers to a 32-bit floating point addition operation of an adder. A 32-bit floating point addition operation can consume 0.9 pj of energy.

[0494] “8b Mult” refers to an 8-bit integer multiplication operation of a multiplier. An 8-bit integer multiplication operation can consume 0.2 pj of energy.

[0495] “32b Mult” refers to a 32-bit integer multiplication operation of a multiplier. A 32-bit integer multiplication operation can consume 3.1 pj of energy.

[0496] “16b FP Mult” refers to a 16-bit floating point multiplication operation of a multiplier. A 16-bit floating point multiplication operation can consume 1.1 pj of energy.

[0497] “32b FP Mult” refers to a 32-bit floating point multiplication operation of a multiplier. A 32-bit floating point multiplication operation can consume 3.7 pj of energy.

[0498] “32b SRAM Read” refers to a 32-bit data read access when the cache memory 322 of the artificial neural network memory system 300 is a static random access memory (SRAM). Reading 32-bit data from the cache memory 322 to the processor 310 can consume 5 pj of energy.

[0499] “32b DRAM Read” refers to a 32-bit data read access when the memory 330 of the artificial neural network memory system 300 is a DRAM. Reading 32-bit data from the memory 330 to the processor 310 can consume 640 pj of energy. The unit of energy refers to a picojoule (pj).

[0500] The difference in energy consumed per unit operation when comparing a 32-bit floating point multiplication and an 8-bit integer multiplication performed by the artificial neural network memory system 300 is approximately 18.5 times. The difference in energy consumed per unit operation when reading 32-bit data from a memory 330 composed of a DRAM and reading 32-bit data from a cache memory 322 composed of an SRAM is approximately 128 times.

[0501] That is, the larger the bit size of data, the greater the power consumption from the perspective of power consumption. In addition, when a floating point operation is used, power consumption increases more than an integer operation. In addition, when data is read from a DRAM, power consumption rapidly increases.

[0502] In an artificial neural network memory system 300 according to yet another exemplary embodiment of the disclosure, the capacity of the cache memory 322 can be configured to be sufficient to store all data values of the artificial neural network model 1300.

[0503] The cache memory according to an exemplary embodiment is not limited to SRAM. Examples of static memory capable of performing a high-speed operation like SRAM include SRAM, MRAM, STT-MRAM, eMRAM, OST-MRAM, etc. In addition, MRAM, STT-MRAM, eMRAM, and OST-MRAM are static memories having a non-volatile characteristic. Thus, when the power supply of the artificial neural network memory system 300 is turned off and then restarted, the artificial neural network model 1300 does not need to be provided from the memory 330 again, but exemplary embodiments according to the disclosure are not limited thereto.

[0504] According to the above-described configuration, when the artificial neural network memory system 300 performs an inference operation of the artificial neural network model 1300 based on the artificial neural network data locality pattern 1400, power consumption due to a read operation of the memory 330 can be significantly reduced.

[0505] Figure 14 is a schematic diagram for explaining an artificial neural network memory system according to various exemplary embodiments of the disclosure.

[0506] Hereinafter, various exemplary embodiments according to the disclosure will be described with reference to Figure 14 Figure 14 The number of various cases in which various exemplary embodiments according to the disclosure can be executed can be explained.

[0507] ​According to various exemplary embodiments of the disclosure, the artificial neural network memory system 400 includes at least one processor, at least one memory, and at least one artificial neural network memory controller ADC configured to include the at least one processor and receive data access requests from the at least one processor to provide the memory access requests to the at least one memory. The at least one artificial neural network memory controller ADC can be configured substantially the same as the exemplary artificial neural network memory controllers 120, 220, and 320. However, it is not limited thereto, and one artificial neural network memory controller of the artificial neural network memory system 400 can be configured differently from the other artificial neural network memory controllers. Hereinafter, for the convenience of description, the repetitive description of the artificial neural network memory controllers 411, 412, 413, 414, 415, 416, and 517 of the artificial neural network memory system and the above-described artificial neural network memory controllers 120, 220, and 320 will be omitted.

[0508] The at least one artificial neural network memory controller is configured to connect the at least one processor and the at least one memory. At this time, in a data transmission path between the at least one processor and the at least one memory, there can be a corresponding artificial neural network data locality. Accordingly, the artificial neural network memory controller located in the data transmission path can be configured to extract a corresponding artificial neural network data locality pattern.

[0509] Each artificial neural network memory controller ADC can be configured to monitor each data access request to generate an artificial neural network data locality pattern. The artificial neural network memory system 400 can be configured to include at least one processor. The at least one processor can be configured to process artificial neural network operations alone or in cooperation with other processors.

[0510] The artificial neural network memory system 400 can be configured to include at least one internal memory. The artificial neural network memory system 400 can be configured to be connected to at least one external memory. The internal memory or the external memory can include a dynamic RAM (DRAM), a high bandwidth memory (HBM), a static RAM (SRAM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a flash memory, a ferroelectric RAM (FRAM), a flash memory, a magnetic RAM (MRAM), a hard disk, a phase change memory device (phase change RAM), etc., but the disclosure is not limited thereto.

[0511] The external memory (External MEM1, External MEM2) or the internal memory (Internal MEM1, Internal MEM2) can communicate with the artificial neural network memory system 400 through a corresponding memory interface (External MEM I / F).

[0512] The processor (Processor 1) can include a bus interface unit (BIU) that communicates with the system bus.

[0513] The artificial neural network memory system 400 can include an external memory interface connected to the external memory (External MEM). The external memory interface transmits a memory access request to at least one external memory of the artificial neural network memory system 400 and can receive data in response to the memory access request from the at least one external memory. The configurations and functions disclosed in the exemplary artificial neural network memory controllers 120, 220, and 320 are distributed to a plurality of artificial neural network memory controllers 411, 412, 413, 414, 415, 416, and 517 that will be disposed at specific locations of the artificial neural network memory system 400. In some exemplary embodiments, the processor can be configured to include the artificial neural network memory controller.

[0514] In some exemplary embodiments, the memory can be a DRAM, and in this case, the artificial neural network memory controller can be configured to be included in the DRAM.

[0515] For example, at least one of the artificial neural network memory controllers 411, 412, 413, 414, 415, 416, and 517 can be configured to include a cache memory. Also, the cache memory can be configured to be included in the processor, the internal memory, and / or the external memory.

[0516] For example, at least one of the artificial neural network memory controllers 411, 412, 413, 414, 415, 416, and 517 can be configured to be distributed in a data transmission path between the memory and the processor.

[0517] For example, the artificial neural network memory controller that can be implemented in the artificial neural network memory system 400 can be configured by one of the following: an independently configured artificial neural network memory controller 411; an artificial neural network memory controller 412 included in a system bus; an artificial neural network memory controller 413 configured as an interface of a processor; an artificial neural network memory controller 414 included in a package block between a memory interface of an internal memory and a system bus; an artificial neural network memory controller included in a memory interface of an internal memory; an artificial neural network memory controller 415 included in an internal memory; an artificial neural network memory controller included in a memory interface corresponding to an external memory; an artificial neural network memory controller 416 included in a package block between an interface of an external memory and a system bus; and / or an artificial neural network memory controller 517 included in an external memory. However, the artificial neural network memory controller according to the exemplary embodiment of the present disclosure is not limited thereto.

[0518] For example, the individual artificial neural network data locality patterns generated by the first artificial neural network memory controller 411 and the second artificial neural network memory controller 412 can be the same as or different from each other.

[0519] In other words, the first artificial neural network memory controller 411 can be configured to connect the first processor Processor 1 and the first internal memory Internal MEM 1 through a system bus. At this time, in a data transmission path between the first processor Processor 1 and the first internal memory Internal MEM 1, there can be a first artificial neural network data locality.

[0520] In this case, the third artificial neural network memory controller 413 is shown in the path. However, this is merely illustrative and the third artificial neural network memory controller 413 can be omitted. That is, when at least one artificial neural network memory controller is disposed between a processor and a memory, an artificial neural network data locality pattern of an artificial neural network model processed by the processor can be generated.

[0521] In other words, the second artificial neural network memory controller 412 can be configured to connect the second processor (Processor 2) and the first external memory External MEM 1. At this time, in a data transmission path between the second processor (Processor 2) and the first external memory External MEM 1, there can be a second artificial neural network data locality.

[0522] For example, a first artificial neural network model processed by a first processor (Processor 1) can be an object recognition model, and a second artificial neural network model processed by a second processor (Processor 2) can be a speech recognition model. Accordingly, the artificial neural network models can be different from each other, and the corresponding artificial neural network data locality patterns can also be different from each other.

[0523] That is, the artificial neural network data locality pattern generated by the artificial neural network memory controllers 411, 412, 413, 414, 415, 416, and 517 can be determined according to the pattern characteristics of the data access requests generated by the corresponding processors.

[0524] That is, even if the artificial neural network memory controllers of the artificial neural network memory system 400 are disposed between arbitrary processors and arbitrary memories, the artificial neural network memory controllers can provide adaptability to generate artificial neural network data locality patterns at the corresponding locations. In other words, when two processors cooperate to process an artificial neural network model in parallel, the artificial neural network data locality pattern of the artificial neural network model can be divided to be allocated to each processor. For example, the convolution operation of the first layer is processed by the first processor, and the convolution operation of the second layer is processed by the second processor to allocate the operation of the artificial neural network model. In this case, even if the artificial neural network model is the same, the artificial neural network data locality of the artificial neural network model processed by the respective processors can be restructured in the unit of data access requests. In this case, each artificial neural network memory controller can provide adaptability to generate an artificial neural network data locality pattern corresponding to the data access requests of the processor processed by the artificial neural network memory controller.

[0525] According to the above-described configuration, even if a plurality of artificial neural network memory controllers are distributed between a plurality of processors and a plurality of memories, the performance of the artificial neural network memory system 400 can be optimized by the neural network data locality pattern generated to suit each case. That is, each artificial neural network memory controller analyzes the artificial neural network data locality at its location to optimize for real-time variable processing of artificial neural network operations.

[0526] In some example embodiments, at least one of the artificial neural network memory controllers 411, 412, 413, 414, 415, 416, and 517 can be configured to confirm at least one information of the number of memories, the type of memory, the effective bandwidth of memory, the memory delay, and the size of memory.

[0527] In some example embodiments, at least one of the artificial neural network memory controllers 411, 412, 413, 414, 415, 416, and 517 can be configured to measure the effective bandwidth of the memory in response to the memory access request. Here, the memory can be at least one memory, and each artificial neural network memory controller can measure the effective bandwidth of the channel in communication with each memory. The effective bandwidth can be calculated by measuring the time and data transfer bit rate at which the artificial neural network memory controller generates the memory access request and the memory access request ends.

[0528] In some example embodiments, at least one of the artificial neural network memory controllers 411, 412, 413, 414, 415, 416, and 517 can be configured to have the necessary bandwidth of at least one memory in response to the memory access request.

[0529] In some example embodiments, the artificial neural network memory system 400 includes a plurality of memories, and at least one artificial neural network memory controller can be configured to measure the effective bandwidth of the plurality of memories.

[0530] In some example embodiments, the artificial neural network memory system 400 includes a plurality of memories, and at least one artificial neural network memory controller can be configured to measure the latency of the plurality of memories.

[0531] That is, at least one artificial neural network memory controller can be configured to perform automatic calibration of the memory connected thereto. The automatic calibration can be configured to be performed at the start of the artificial neural network memory system or at a certain period. At least one artificial neural network memory controller can be configured to collect information such as the number of memories connected thereto, the type of memory, the effective bandwidth of the memory, the latency of the memory, and the size of the memory, etc. through the automatic calibration.

[0532] According to the above configuration, the artificial neural network memory system 400 can know the latency and the effective bandwidth of the memory corresponding to the artificial neural network memory controller.

[0533] According to the above configuration, even if independent artificial neural network memory controllers are connected to the system bus, artificial neural network data locality of an artificial neural network model being processed by a processor is generated to control the memory.

[0534] In some example embodiments, at least one artificial neural network memory controller of the artificial neural network memory system 400 can be configured to calculate a time taken to repeat the artificial neural network data locality pattern once and a data size for calculating an effective bandwidth required for the artificial neural network operation. Specifically, when the data access requests contained in the artificial neural network data locality pattern are all processed, the processor completes the inference of the artificial neural network model. The artificial neural network memory system 400 can be configured to measure the time taken for one inference based on the artificial neural network data locality pattern to calculate the number of inferences per second (IPS: inferences per second). In addition, the artificial neural network memory system 400 can be provided with information on the target number of inferences per second from the processor. For example, a specific application requires 30 IPS as the inference rate of a specific artificial neural network model. If the measured IPS is lower than the target IPS, the artificial neural network memory controller 400 can be configured to operate to increase the artificial neural network model processing speed of the processor.

[0535] In some example embodiments, the artificial neural network memory system 400 can be configured to include a system bus configured to control communication of the artificial neural network memory controller, the processor, and the memory. Further, at least one artificial neural network memory controller can be configured to have a master right of the system bus.

[0536] In other words, the artificial neural network memory system 400 can not be a dedicated device for the artificial neural network operation. In this case, the system bus of the artificial neural network memory system 400 can be connected to various peripheral devices, such as a wifi device, a display, a camera, or a microphone. In this case, the artificial neural network memory system 400 can be configured to control the bandwidth of the system bus to stabilize the artificial neural network operation.

[0537] In some example embodiments, at least one artificial neural network memory controller can operate to prioritize the artificial neural network operation over the processing time of the memory access request and to process operations other than the artificial neural network operation at other times.

[0538] In some example embodiments, at least one artificial neural network memory controller can be configured to secure the effective bandwidth of the system bus until at least one memory completes the memory access request.

[0539] In some example embodiments, at least one artificial neural network memory controller is disposed in a system bus, and the system bus can be configured to dynamically change the bandwidth of the system bus based on the artificial neural network data locality pattern generated in the system bus.

[0540] In some example embodiments, at least one artificial neural network memory controller is disposed in a system bus, and the at least one artificial neural network memory controller can be configured to increase a control authority of the system bus to be higher than a control authority when there is no memory access request, until at least one memory completes a response to the memory access request.

[0541] In some example embodiments, at least one artificial neural network memory controller can be configured to set a priority of a data access request of a processor processing artificial neural network operations in a plurality of processors to be higher than a priority of a data access request of a processor processing operations other than the artificial neural network operations.

[0542] In some example embodiments, an artificial neural network memory controller can be configured to directly control a memory.

[0543] In some example embodiments, an artificial neural network memory controller is included in a memory, and the artificial neural network memory controller can be configured to generate at least one access queue. The artificial neural network memory controller can be configured to separately generate an access queue dedicated to artificial neural network operations.

[0544] In some example embodiments, at least one of a plurality of memories can be a DRAM. In this case, at least one artificial neural network memory controller can be configured to re-adjust an access queue of a memory access request. The access queue re-adjustment can be an access queue reordering.

[0545] In some example embodiments, an artificial neural network memory controller can be configured to include an access queue of a plurality of memory access requests. In this case, a first access queue can be an access queue dedicated to artificial neural network operations, and a second access queue can be an access queue for operations other than the artificial neural network operations. The artificial neural network memory controller can be configured to provide data by selecting each access queue according to a priority setting.

[0546] In some example embodiments, at least one artificial neural network memory controller can be configured to calculate a specific bandwidth required for a system bus to process a specific memory access request based on an artificial neural network data locality pattern, and the at least one artificial neural network memory controller can be configured to control an effective bandwidth of the system bus based on the specific bandwidth.

[0547] According to the above-described configuration, the artificial neural network memory system 400 can be configured to lower a priority of a memory access request of various peripheral devices or increase a priority of an early data access request based on an artificial neural network data locality pattern.

[0548] According to the above configuration, the artificial neural network memory controller re-adjusts the processing order of the data access requests of the system bus to fully utilize the bandwidth of the system bus while processing the artificial neural network operation, and to generate the bandwidth of processing data of other peripherals when there is no artificial neural network operation.

[0549] According to the above configuration, the artificial neural network memory controller can re-adjust the processing order of the data access requests based on the artificial neural network data locality pattern. In addition, the artificial neural network memory controller re-adjusts the priority based on the identification information contained in the data access request. That is, the effective bandwidth of the system bus is dynamically changed from the perspective of the artificial neural network operation, so that the effective bandwidth can be improved. Therefore, the operation efficiency of the system bus can be improved. Therefore, from the perspective of the artificial neural network memory controller, the effective bandwidth of the system bus can be improved.

[0550] In some exemplary embodiments, at least one artificial neural network memory controller can be configured to perform machine learning of the data access requests. That is, at least one artificial neural network memory controller can further include an artificial neural network model configured to machine learn the artificial neural network data locality pattern. That is, the artificial neural network data locality pattern is machine learned in order to learn a certain pattern (i.e., interrupt another data access request in the process of processing the data access request according to the actual artificial neural network data locality) to make a prediction.

[0551] When the early data access request is generated, the artificial neural network model embedded in the artificial neural network memory controller can be machine trained to raise the control right of the system bus to be higher than when the early data access request is not generated.

[0552] In some exemplary embodiments, at least one artificial neural network memory controller further includes a plurality of hierarchical cache memories, and at least one artificial neural network memory controller can be configured to perform machine learning of the data access requests between layers of the plurality of hierarchical cache memories.

[0553] In some exemplary embodiments, at least one artificial neural network memory controller can be configured to have at least one of effective bandwidth, power consumption, and delay information of each layer of the plurality of hierarchical cache memories.

[0554] According to the above configuration, the artificial neural network memory controller can be configured to generate an artificial neural network data locality pattern through machine learning, and the machine-learned artificial neural network data locality pattern can increase the probability of predicting the occurrence of a specific pattern when various data access requests unrelated to artificial neural network operations are generated in a specific pattern. Further, the characteristics of various artificial neural network models and other operations processed by the processor are predicted through reinforcement learning to improve the efficiency of artificial neural network operations.

[0555] In some example embodiments, at least one artificial neural network memory controller can be configured to divide and store data to be stored in a plurality of memories based on effective bandwidth and latency of each of the plurality of memories.

[0556] For example, data is configured by a bit group of L bits, and the plurality of memories includes a first memory and a second memory. The first memory is configured to divide and store M-bit data from the bit group of L bits based on a first effective bandwidth or a first latency, and the second memory is configured to divide and store N-bit data from the bit group of L bits based on a second effective bandwidth or a second latency. The sum of M bits and N bits can be configured to be less than or equal to L bits. Further, the plurality of memories further includes a third memory, and the third memory is configured to store O-bit data from the bit group of L bits based on a third effective bandwidth or a third latency, and the sum of M bits, N bits, and O bits can be configured to be equal to L bits.

[0557] For example, data is configured by a bit group of L bits, and the plurality of memories includes a first memory and a second memory. The first memory is configured to divide and store M-bit data from the bit group of L bits based on a first effective bandwidth or a first latency, and the second memory is configured to divide and store N-bit data from the bit group of L bits based on a second effective bandwidth or a second latency. The sum of M bits and N bits can be configured to be less than or equal to L bits. Further, the plurality of memories further includes a third memory, and the third memory is configured to store O-bit data from the bit group of L bits based on a third effective bandwidth or a third latency, and the sum of M bits, N bits, and O bits can be configured to be equal to L bits.

[0558] According to the above configuration, when the bandwidth of one memory is low, the artificial neural network memory controller can allocate data to be stored or read, so that the effective bandwidth of the memory can be improved. For example, the artificial neural network memory controller can be configured to divide the 8-bit quantized weight value to store or read 4 bits in a first memory and 4 bits in a second memory. Thus, from the perspective of the artificial neural network memory controller, the effective bandwidth of the memory can be improved.

[0559] The artificial neural network memory controller can be configured to further include a cache memory configured to merge and store data divided to be stored in the plurality of memories. That is, at least one artificial neural network memory controller further includes a cache memory, and can be configured to merge data distributed to be stored in the plurality of memories to store the merged data in the cache memory. Accordingly, the merged data can be provided to the processor.

[0560] To merge the divided data, at least one artificial neural network memory controller can be configured to store division information of data divided to be stored in the plurality of memories. Various exemplary embodiments of the disclosure will be described as follows.

[0561] According to one exemplary embodiment of the disclosure, an artificial neural network memory system can be configured to include at least one processor configured to generate a data access request corresponding to an artificial neural network operation, and at least one artificial neural network memory controller configured to generate an artificial neural network data locality pattern of the artificial neural network operation by sequentially recording the data access request, and generate an advance data access request predicting a next data access request of the data access request generated by the at least one processor based on the artificial neural network data locality pattern. Here, the artificial neural network data locality is artificial neural network data locality reconstructed at a processor-memory level.

[0562] According to an exemplary embodiment of the disclosure, an artificial neural network memory system can be configured to include at least one processor configured to process an artificial neural network model, and at least one artificial neural network memory controller configured to store artificial neural network data locality information of the artificial neural network model, and predict data to be requested by the at least one processor based on the artificial neural network data locality information to generate an advance data access request.

[0563] The artificial neural network memory system can be configured to further include at least one memory and a system bus configured to control communication of the artificial neural network memory controller, the at least one processor, and the at least one memory. According to an exemplary embodiment of the disclosure, the artificial neural network memory system includes a processor, a memory, and a cache memory, and is configured to generate an advance data access request including data to be requested by the processor based on artificial neural network data locality information, and store data corresponding to the advance data access request from the memory in the cache memory before the processor requests.

[0564] According to example embodiments of the present disclosure, an artificial neural network memory system can be configured to operate in either of a first mode configured to operate by receiving artificial neural network data locality information and a second mode configured to operate by observing data access requests generated by a processor to predict artificial neural network data locality information.

[0565] At least one artificial neural network memory controller can be configured to further generate a look-ahead data access request in sequence based on artificial neural network data locality patterns.

[0566] At least one artificial neural network memory controller can be configured to generate a look-ahead data access request before generating a next data access request.

[0567] At least one processor can be configured to send a data access request to at least one artificial neural network memory controller.

[0568] At least one artificial neural network memory controller can be configured to output a look-ahead data access request in response to a data access request.

[0569] The data access request can be configured to further include a memory address.

[0570] The data access request can be configured to further include a start address and an end address of a memory.

[0571] At least one artificial neural network memory controller can be configured to generate a memory access request based on one of a data access request generated by at least one processor and a look-ahead data access request generated by the artificial neural network memory controller.

[0572] The data access request can be configured to further include a start address of a memory and a contiguous data read trigger.

[0573] The data access request can be configured to further include a start address of a memory and a number information of contiguous data.

[0574] The data access request and the look-ahead data access request can be configured to further include a data access request token of the same matching memory address.

[0575] The data access request can be configured to further include identification information to identify whether it is a memory read command or a write command.

[0576] The data access request can be configured to further include identification information to identify whether it is a memory overwrite command.

[0577] The data access request can be configured to further include identification information to identify whether it is inference data, weight data, or feature map data.

[0578] The data access request can be configured to further include identification information to identify whether it is learning data or evaluation data.

[0579] The data access request can be configured to further include identification information to identify whether the artificial neural network operation is an operation for learning or an operation for inference.

[0580] When the at least one processor generates the next data access request, the at least one artificial neural network memory controller can be configured to determine whether the previous data access request and the next data access request are the same request.

[0581] When the previous data access request and the next data access request are the same request, the at least one artificial neural network memory controller can be configured to maintain the artificial neural network data locality pattern.

[0582] When the previous data access request and the next data access request are different, the at least one artificial neural network memory controller can be configured to update the artificial neural network data locality pattern.

[0583] The artificial neural network data locality pattern can be configured to further include data in which addresses of a memory in which the data access requests are recorded in order.

[0584] The at least one artificial neural network memory controller can be configured to generate the artificial neural network data locality pattern by detecting a repetitive pattern of memory addresses included in the data access requests.

[0585] The artificial neural network data locality pattern can be configured by a memory address having a repetitive cycle characteristic.

[0586] The artificial neural network data locality pattern can be configured to further include identification information for identifying a start and an end of an operation of an artificial neural network model.

[0587] The at least one processor can be configured to be provided with data corresponding to the data access request from the artificial neural network memory controller.

[0588] The at least one artificial neural network memory controller can be configured to further include an artificial neural network model configured to machine-learn the artificial neural network data locality pattern.

[0589] At least one artificial neural network memory controller can be configured to store an update pattern and an advance pattern of artificial neural network data locality patterns to determine whether an artificial neural network model is changed.

[0590] At least one artificial neural network memory controller can be configured to determine whether a data access request is a request for one artificial neural network model or a mixture of requests for a plurality of artificial neural network models.

[0591] When there are a plurality of artificial neural network models, at least one artificial neural network memory controller can be configured to further generate artificial neural network data locality patterns corresponding to the number of artificial neural network models.

[0592] At least one artificial neural network memory controller can be configured to individually generate a corresponding advance data access request based on the artificial neural network data locality pattern.

[0593] At least one artificial neural network memory controller can be configured to further generate a data access request corresponding to the data access request.

[0594] At least one artificial neural network memory controller can be configured to further generate a data access request corresponding to the advance data access request.

[0595] Each of the data access request, the advance data access request, and the memory access request can be configured to include a corresponding memory address value and an operation pattern.

[0596] At least one artificial neural network memory controller can be configured to further generate a memory access request including at least a part of information included in the data access request and the advance data access request.

[0597] At least one memory configured to communicate with at least one artificial neural network memory controller is also included, and the at least one memory can be configured to operate in response to a memory access request output from the at least one artificial neural network memory controller.

[0598] At least one memory can be configured to store at least one of inference data, weight data, and feature map data.

[0599] At least one neural network artificial neural network memory controller can be configured to further include a cache memory configured to store data transmitted from the at least one memory in response to the memory access request.

[0600] When the at least one processor outputs the next data access request, the at least one artificial neural network memory controller determines whether the early data access request and the next data access request are the same request. If the early data access request and the next data access request are the same, the at least one artificial neural network memory controller can be configured to provide the data stored in the cache memory to the at least one processor, and if the early data access request and the next data access request are different, the at least one artificial neural network memory controller can be configured to generate a new memory access request based on the next data access request.

[0601] The at least one artificial neural network memory controller sequentially generates the at least one memory access request based on a remaining capacity of the cache memory to minimize the remaining capacity of the cache memory.

[0602] The at least one artificial neural network memory controller can be configured to measure an effective bandwidth of the at least one memory in response to the memory access request.

[0603] The at least one artificial neural network memory controller can be configured to have a necessary bandwidth of the at least one memory in response to the memory access request.

[0604] The at least one artificial neural network memory controller can be configured to measure an inference per second (IPS) of the artificial neural network operation by calculating a number of repetitions of the artificial neural network data locality pattern within a certain time.

[0605] The at least one artificial neural network memory controller can be configured to calculate a time taken to repeat the artificial neural network data locality pattern once and a data size for calculating an effective bandwidth required for the artificial neural network operation.

[0606] The at least one memory further includes a DRAM including a refresh function to update a voltage of a memory cell, and the at least one artificial neural network memory controller can be configured to selectively control a refresh of a memory address region of the at least one memory corresponding to the memory access request in correspondence with the early data access request.

[0607] The at least one memory further includes a pre-charge function to charge a global bit line of the memory with a certain voltage, and the at least one artificial neural network memory controller can be configured to selectively provide a pre-charge to a memory address region of the at least one memory corresponding to the memory access request in correspondence with the early data access request.

[0608] The at least one memory further includes a plurality of memories, and the at least one artificial neural network memory controller can be configured to measure an effective bandwidth of the plurality of memories, respectively.

[0609] The at least one memory further includes a plurality of memories, and the at least one artificial neural network memory controller can be configured to measure a latency of the plurality of memories, respectively.

[0610] The at least one memory further includes a plurality of memories, and the at least one artificial neural network memory controller can be configured to divide and store data to be stored in the plurality of memories based on an effective bandwidth and a latency of each of the plurality of memories.

[0611] The data is configured by a group of L bits, and the plurality of memories further includes a first memory and a second memory. The first memory is configured to divide and store M-bit data from the group of L bits based on a first effective bandwidth or a first latency, and the second memory is configured to divide and store N-bit data from the group of L bits based on a second effective bandwidth or a second latency. A sum of the M bits and the N bits can be configured to be less than or equal to the L bits.

[0612] The plurality of memories further includes a third memory, and the third memory is configured to store O-bit data from the group of L bits based on a third effective bandwidth or a third latency, and a sum of the M bits, the N bits, and the O bits can be configured to be equal to the L bits.

[0613] The at least one artificial neural network memory controller can be configured to further include a cache memory configured to merge and store the data divided to be stored in the plurality of memories.

[0614] The data is configured by P data packets, and the plurality of memories includes a first memory and a second memory. The first memory is configured to store R data packets of the P data packets based on a first effective bandwidth or a first latency, and the second memory is configured to store S data packets of the P data packets based on a second effective bandwidth or a second latency. A sum of the R and the S can be configured to be less than or equal to the P.

[0615] The plurality of memories further includes a third memory, and the third memory is configured to store T data packets of the P data packets based on a third effective bandwidth or a third latency, and a sum of the R, the S, and the T can be configured to be equal to the P.

[0616] The at least one memory further includes a plurality of memories, and the at least one artificial neural network memory controller further includes a cache memory and is configured to merge data distributed to be stored in the plurality of memories to store the merged data in the cache memory.

[0617] The at least one memory further includes a plurality of memories, and the at least one artificial neural network memory controller can be configured to store division information of the data divided to be stored in the plurality of memories.

[0618] The at least one artificial neural network memory controller can be configured to store a portion of data in the cache memory as much as a latency based on the early data access request and the latency of the at least one memory.

[0619] The at least one artificial neural network memory controller can be configured to store a portion of data in the cache memory based on the early data access request and a required data bandwidth of the at least one memory.

[0620] When the at least one processor generates a next data access request, the at least one artificial neural network memory controller first provides data stored in the cache memory and controls remaining data from the at least one memory in a read burst mode to reduce latency of the at least one memory.

[0621] When the at least one processor generates a next data access request based on the early data access request and a latency value of the at least one memory, the at least one artificial neural network memory controller starts a read burst mode of the at least one memory as much as the latency value in advance to reduce latency of the at least one memory.

[0622] A system bus configured to control communication of the artificial neural network memory controller, the at least one processor, and the at least one memory can be further included.

[0623] The at least one artificial neural network memory controller can be configured to have a master right of the system bus.

[0624] The at least one artificial neural network memory controller further includes an artificial neural network model, and when the early data access request is generated, the artificial neural network model can be machine-trained to increase a right of the system bus higher than a right when the early data access request is not generated.

[0625] The at least one artificial neural network memory controller can be configured to secure an effective bandwidth of the system bus until the at least one memory completes a memory access request.

[0626] The at least one artificial neural network memory controller can be configured to calculate a specific bandwidth required for the system bus to process a specific memory access request based on an artificial neural network data locality pattern, and the at least one artificial neural network memory controller can be configured to control an effective bandwidth of the system bus according to the specific bandwidth.

[0627] The at least one artificial neural network memory controller is disposed in a system bus, and the system bus is configured to dynamically change a bandwidth of the system bus based on an artificial neural network data locality pattern generated in the system bus.

[0628] At least one artificial neural network memory controller can operate to prioritize artificial neural network operations over processing of memory access requests at processing time and to process operations other than artificial neural network operations at other times.

[0629] At least one artificial neural network memory controller and at least one processor can be configured to directly communicate with each other.

[0630] The artificial neural network memory controller can be configured to further include a first access queue that is an access queue dedicated to artificial neural network operations and a second access queue that is an access queue other than artificial neural network operations, and the artificial neural network memory controller can be configured to select an access queue to provide data according to a priority setting.

[0631] At least one artificial neural network memory controller further includes a plurality of hierarchical cache memories, and the at least one artificial neural network memory controller can be configured to further include an artificial neural network model configured to perform machine learning of data access requests between layers of the plurality of hierarchical cache memories.

[0632] At least one artificial neural network memory controller can be configured to further be provided with at least one of effective bandwidth, power consumption, and delay information of each layer of the plurality of hierarchical cache memories.

[0633] Including: at least one processor configured to generate a data access request corresponding to an artificial neural network operation; at least one artificial neural network memory controller configured to store artificial neural network data locality patterns of artificial neural network operations generated from a compiler and to generate an advance data access request that predicts a next data access request of the data access request generated by the at least one processor based on the artificial neural network data locality patterns; and at least one memory configured to communicate with the at least one artificial neural network memory controller. The at least one memory can be configured to operate according to a memory access request output from the at least one artificial neural network memory controller.

[0634] At least one artificial neural network memory system can be configured to further include at least one memory and a system bus configured to control communication of the artificial neural network memory controller, the at least one processor, and the at least one memory.

[0635] At least one artificial neural network memory controller is disposed in the system bus, and at least one artificial neural network memory controller can be configured to raise a control authority of the system bus to be higher than a control authority when there is no memory access request until at least one memory completes a response to the memory access request.

[0636] At least one artificial neural network memory controller includes one or more artificial neural network memory controllers configured to be included in a DRAM.

[0637] At least one artificial neural network memory controller includes one or more artificial neural network memory controllers configured to be included in at least one processor.

[0638] At least one memory further includes a DRAM or at least one memory is a DRAM, and at least one artificial neural network memory controller can be configured to re-adjust an access queue of a memory access request. That is, at least one artificial neural network memory controller can be configured to control a reordering clue of a memory controller of a DRAM.

[0639] The memory access request related to an artificial neural network operation provided from the artificial neural network memory controller to the memory controller of the memory can further include priority information interpretable by the memory controller of the memory.

[0640] According to the above configuration, the memory controller of the memory can be configured to reorder a memory access queue in the memory controller based on the priority information included in the memory access request generated by the artificial neural network memory controller, regardless of whether the memory access request is related to an artificial neural network operation. Accordingly, an access queue for processing a memory access request for an artificial neural network operation can be processed earlier than an access queue for another type of memory access request. Accordingly, the artificial neural network memory controller can increase an effective bandwidth of the corresponding memory.

[0641] The order of processing the memory access request determined by the memory controller of the DRAM can be re-adjusted by the priority information provided by the artificial neural network memory controller.

[0642] For example, when the priority of the memory access request generated by the artificial neural network memory controller is set to urgent, the memory controller of the DRAM can change the processing sequence of the memory access request to the first priority.

[0643] The artificial neural network memory controller can be configured to generate at least one access queue.

[0644] The at least one memory includes an artificial neural network memory controller, and the artificial neural network memory controller can be configured to generate an access queue dedicated to artificial neural network operations alone.

[0645] The at least one artificial neural network memory controller can be configured to re-adjust an access queue of a memory access request.

[0646] The at least one memory further includes a read burst function, and the at least one artificial neural network memory controller can be configured to set a storage area of the at least one memory in consideration of the read burst function.

[0647] The at least one memory further includes a read burst function, and the at least one artificial neural network memory controller can be configured to process a write operation in a storage area of the at least one memory in consideration of the read burst function.

[0648] The at least one processor further includes a plurality of processors, and the at least one artificial neural network memory controller can be configured to set a priority of a data access request of a processor processing artificial neural network operations among the plurality of processors to be higher than a priority of a data access request of a processor processing operations other than the artificial neural network operations.

[0649] The features, structures, effects, and the like described in the foregoing embodiments are included in one embodiment of the disclosure and are not necessarily limited to one embodiment. Furthermore, a person skilled in the art can combine or modify the features, structures, effects, and the like shown in each embodiment for other embodiments to be performed. Therefore, combinations and modifications of the disclosure are interpreted to be included in the scope of the disclosure.

[0650] In the above description, the disclosure has been described based on exemplary embodiments, but the exemplary embodiments serve to illustrate rather than limit the present invention, and a person skilled in the art should understand that various modifications and applications not exemplified in the above description can be made without departing from the scope of the essential features of the exemplary embodiments. For example, each constituent element specifically present in the exemplary embodiments can be modified and implemented. Furthermore, differences related to the modifications and applications should be understood to be included in the scope of the present invention defined by the appended claims.

[0651] [National R&D Project Supporting the Invention]

[0652] [Project Unique Number] 1711117009

[0653] [Task Number] 2020-0-01303-001

[0654] [Ministry of Science and ICT]

[0655] [Project management (professional) agency name] Information and Communication Planning and Evaluation Institute

[0656] [Research project name] Next-generation intelligent semiconductor technology development (design) (R&D)

[0657] [Research project name] High-performance AI server HBM3 level or above interface technology development

[0658] [Contribution rate] 1 / 1

[0659] [Project execution agency name] DeepX Co., Ltd.

[0660] [Research period] 2020.04.01 ~ 2020.12.31

Claims

1. An artificial neural network memory system comprising: at least one processor configured to generate data access requests to access data required by the processor to process an artificial neural network model; and at least one artificial neural network memory controller configured to sequentially record the data access requests to generate an artificial neural network data locality pattern of the artificial neural network model, wherein the artificial neural network data locality pattern is sequence information of word units of data required by the processor to process the artificial neural network model, wherein the sequence corresponds to a sequence of operations comprising read operations and write operations for the artificial neural network model, and wherein the sequence of operations is determined at the time of compiling the artificial neural network model, and based on the artificial neural network data locality pattern, generate an advance data access request that predicts a next data access request of the data access requests generated by the at least one processor.

2. The artificial neural network memory system of claim 1, wherein, The at least one artificial neural network memory controller is further configured to sequentially generate the advance data access request based on the artificial neural network data locality pattern.

3. The artificial neural network memory system of claim 1, wherein, The at least one artificial neural network memory controller is configured to generate the advance data access request prior to generating the next data access request.

4. The artificial neural network memory system of claim 1, wherein, The data access request is further configured to include a start address of the memory and a continuous data read trigger.

5. The artificial neural network memory system of claim 1, wherein, When the at least one processor generates the next data access request, the at least one artificial neural network memory controller is configured to determine whether the advance data access request and the next data access request are the same request.

6. The artificial neural network memory system of claim 1, wherein, The artificial neural network data locality pattern is constituted by memory addresses having a repetitive loop characteristic.

7. The artificial neural network memory system of claim 1, wherein, The at least one artificial neural network memory controller is configured to store an updated pattern and a previous pattern of the artificial neural network data locality pattern to determine whether the artificial neural network model is changed.

8. The artificial neural network memory system of claim 1, wherein, The at least one artificial neural network memory controller is configured to determine whether the data access request is a request of one artificial neural network model or a mixture of requests of multiple artificial neural network models.

9. The artificial neural network memory system of claim 1, wherein, When there are multiple artificial neural network models, the at least one artificial neural network memory controller is further configured to: generate artificial neural network data locality patterns corresponding to a number of artificial neural network models, and generate corresponding advance data access requests based on the artificial neural network data locality patterns.

10. The artificial neural network memory system of claim 1, further comprising: at least one memory configured to communicate with the at least one artificial neural network memory controller, wherein the at least one memory is configured to operate in response to a memory access request output from the at least one artificial neural network memory controller.

11. The artificial neural network memory system of claim 10, wherein, The at least one artificial neural network memory controller is further configured to include a cache memory configured to store data transmitted from the at least one memory in response to the memory access request.

12. The artificial neural network memory system of claim 11, wherein, The at least one artificial neural network memory controller is configured to generate at least one of the memory access requests in sequence based on a remaining capacity of the cache memory to minimize the remaining capacity of the cache memory.

13. The artificial neural network memory system of claim 10, wherein, The at least one artificial neural network memory controller is configured to measure an effective bandwidth of the at least one memory in response to the memory access requests.

14. The artificial neural network memory system of claim 10, wherein, The at least one memory further comprises a DRAM having a refresh function to update a voltage of a memory cell, and wherein the at least one artificial neural network memory controller is configured to selectively control a refresh of a memory address region of the at least one memory corresponding to the memory access requests in response to the early data access request.

15. The artificial neural network memory system of claim 10, wherein, The at least one memory further comprises a pre-charge function to charge a global bit line of the memory with a specific voltage, and wherein the at least one artificial neural network memory controller is configured to selectively provide a pre-charge to a memory address region of the at least one memory corresponding to the memory access requests in response to the early data access request.

16. The artificial neural network memory system of claim 10, wherein The at least one memory further comprises a plurality of memories, and wherein the at least one artificial neural network memory controller is configured to store division information of the data divided to be stored in the plurality of memories.

17. The artificial neural network memory system of claim 10, further comprising: a system bus configured to control a communication of the artificial neural network memory controller, the at least one processor, and the at least one memory, wherein the at least one artificial neural network memory controller is configured to have a master right of the system bus.

18. The artificial neural network memory system of claim 1, wherein the at least one artificial neural network memory controller and the at least one processor are configured to directly communicate with each other.

19. The artificial neural network memory system of claim 10, wherein The at least one artificial neural network memory controller further comprises a plurality of hierarchical cache memories, and wherein the at least one artificial neural network memory controller is configured to further comprise an artificial neural network model configured to perform a machine learning of the data access requests among layers of the plurality of hierarchical cache memories.

20. An artificial neural network memory system, comprising: at least one processor configured to process an artificial neural network model; and at least one artificial neural network memory controller configured to: storing artificial neural network data locality information of the artificial neural network model, wherein the artificial neural network data locality information is sequence information of word units of data required by the processor to process the artificial neural network model, wherein the sequence corresponds to a sequence of operations including read operations and write operations for the artificial neural network model, and wherein the sequence of operations is determined when the artificial neural network model is compiled, and predicting data to be requested by the at least one processor to process the artificial neural network model based on the artificial neural network data locality information to generate early data access requests.

21. The artificial neural network memory system of claim 20, further comprising: at least one memory; and a system bus configured to control communication of the artificial neural network memory controller, the at least one processor, and the at least one memory.

22. The artificial neural network memory system of claim 21, wherein the at least one artificial neural network memory controller is disposed in the system bus, and wherein the at least one artificial neural network memory controller is configured to raise a control authority of the system bus to be higher than a control authority when there is no memory access request until the at least one memory completes a response to the memory access request.

23. The artificial neural network memory system of claim 20, further comprising: a DRAM, wherein the at least one artificial neural network memory controller includes one or more artificial neural network memory controllers configured to be included in the DRAM.

24. The artificial neural network memory system of claim 20, wherein, the at least one artificial neural network memory controller includes one or more artificial neural network memory controllers configured to be included in the at least one processor.

25. The artificial neural network memory system of claim 20, further comprising: a DRAM, wherein the at least one artificial neural network memory controller is configured to re-adjust an access queue of the memory access request.

26. The artificial neural network memory system of claim 20, further comprising: at least one memory further including a read burst function, wherein the at least one artificial neural network memory controller is configured to set a storage area of the at least one memory in consideration of the read burst function.

27. An artificial neural network memory system, comprising: a processor, a memory, and a cache memory in a memory controller, wherein an early data access request is generated based on artificial neural network data locality information, the early data access request comprising data to be requested by the processor to process an artificial neural network model, wherein the artificial neural network data locality information is sequence information of word units of data required by the processor to process the artificial neural network model, wherein the sequence corresponds to a sequence of operations comprising read operations and write operations for the artificial neural network model, wherein the sequence of operations is determined at the time of compilation of the artificial neural network model, and wherein data from the memory corresponding to the early data access request is stored in the cache memory prior to a request by the processor.

28. The artificial neural network memory system of claim 27, wherein the artificial neural network memory system is configured to operate in any one of the following modes: a first mode configured to operate by receiving the artificial neural network data locality information, and a second mode configured to operate by observing the data access requests produced by the processor to predict the artificial neural network data locality information.

Images