Data caching device based on data-address sequence information of artificial neural network

Abstract

A data caching or data management device for caching data between at least one processor and at least one memory and supporting artificial neural network (ANN) operations performed by the at least one processor is disclosed. The data caching device or the data management device may include an internal controller to predict a next data operation request based on ANN data locality of the ANN operation. The internal controller monitors data operation requests associated with the ANN operation from among data operation requests actually made between the at least one processor and the at least one memory, thereby extracting ANN data locality of the ANN operation.

Description

This application is a divisional application of the invention patent application with application number 202080007589.1, application date June 4, 2020, and invention name "Data management device based on artificial neural network data location using data cache to support high-speed artificial neural network operation". Technical Field

[0001] The present disclosure relates to a data management device for supporting artificial neural network operations, and more particularly to a data management method based on the data location of the artificial neural network and using data cache to support high-speed artificial neural network operations, as well as a device applying the method. Background Art

[0002] There is increasing interest in artificial intelligence using artificial neural networks (ANNs), which can learn from data to infer the correlations of parameters inherent in a data set.

[0003] Recently, convolutional neural networks (CNNs) have been widely used in artificial intelligence fields, such as image recognition and self-driving cars.

[0004] A convolutional neural network consists of an input layer, an output layer, and one or more inner layers between them.

[0005] The output layer includes one or more neurons, and each of the input layer and the inner layer includes a plurality of neurons.

[0006] Neurons contained in adjacent layers are connected in various ways through synapses, and a weight is assigned to each synapse.

[0007] The values ​​of the neurons contained in the input layer are determined according to the input signal such as the image to be recognized.

[0008] The values ​​of the neurons contained in the inner and output layers are calculated based on the neurons and synapses contained in the previous layer.

[0009] In a convolutional neural network connected in this way, the weights of the synapses are determined through training operations.

[0010] For example, an inference operation is performed using a trained convolutional neural network to perform image recognition. For example, an input image determines the values ​​of multiple neurons in the input layer, then passes through the inner layer through multiple operations, and outputs the recognition result to the neurons in the output layer.

[0011] Many operations must be performed both during the training operation of a convolutional neural network and during the inference operation using a convolutional neural network.

[0012] Furthermore, many memory access operations are performed to store temporary data generated during an operation or to read stored temporary data.

[0013] Generally, there is a problem in that overall operation performance is degraded due to the time required for data input / output operations between a processor and a memory that perform a neural network operation.

[0014] In order to improve the problems of the prior art, a memory processing technology that includes a neural network processor in a memory has been proposed. For example, KR 10-2019-0018888 "Memory device including a neural network processing circuit and a storage system including the device" corresponds to this technology.

[0015] In addition, as a neural network processor (NNP) that supports artificial neural network operations, U.S. Patent No. 9,805,304 “prefetching weights for use in a neural network processor” discloses a configuration for optimizing an artificial neural network by prefetching network weights for a neural network layer along a first dimension of a systolic array during an artificial neural network operation.

[0016] In the prior art, KR 10-2019-0018888 is motivated by the need to shorten the time required to send and receive data by including a neural network processing circuit determined in a memory. In addition, in the prior art, US 9,805,304 discloses a neural network processor (NNP) dedicated to a specific artificial neural network, the neural network processor including a sequencer that performs a next data operation based on structural information of a predetermined artificial neural network.

[0017] However, recently, artificial neural network technology is trying to improve performance through various types of networks or multiple combinations of dynamically deformable networks. Therefore, by only trying to process neural network processing operations on the memory side or on a neural network dedicated to a specific artificial neural network, and optimizing a weight sequencer based on a predetermined artificial neural network in a neural network processor (NNP) dedicated to a certain neural network, the application of various artificial neural network technologies may be limited in terms of application scope. Summary of the invention Technical issues

[0018] In a general computing system, although the performance of a processor that performs an operation is excellent, the performance of a memory that provides data that is the object of the operation is insufficient compared to the performance of the processor. As a result, the overall operating performance of the system is reduced. In particular, in the operation of artificial intelligence / artificial neural networks that calculate big data, imbalance is a more serious problem. Therefore, it is recognized that the problem of artificial neural network operation needs to be improved to the greatest extent.

[0019] Since artificial neural network operations are characterized by data-intensive processing, the overall operation performance is greatly affected by the bandwidth and waiting time of the memory device that stores the data. This is because, even when the processor has excellent operation capabilities, if the input data required for the operation cannot be retrieved from the memory at the appropriate time, the processor cannot perform actual operations and must wait in an idle state.

[0020] Depending on the type of memory, when the processor sends a memory address value to read the data required for the operation from the memory, the actual requested data is output from the memory after a waiting time of several clock cycles. In the case of DRAM, it takes several clocks to activate the word lines and bit lines according to the row address, several clocks to activate the column lines, and several clocks to pass a path to transfer the data outside the memory. In the case of NAND flash memory, since the cells activated at one time are large, it may take several additional clocks to search for the data at the required address.

[0021] If the processor does not have the data required for the operation, and it takes several clocks to access the processor's external memory for data reading and actually read the data, the operation will stop because there is no necessary data at that time, resulting in a decrease in overall operation performance.

[0022] The purpose of the present disclosure is to improve the problems of the prior art and to provide a data management device capable of minimizing the limitation when selecting / modifying an artificial neural network by executing an artificial neural network processing operation in at least one processor and capable of expanding the scope of application. In addition, in an artificial neural network operating system using various memory devices (DRAM, SRAM, flash memory, and NVRAM), it is intended to maximize the bandwidth of a given memory by maximizing the waiting time for reading data from the memory used for artificial neural network operation.

[0023] An object of the present disclosure is to provide a data management device that can support high-speed artificial neural network operations even at any location between at least one processor and a memory device.

[0024] An object of the present disclosure is to minimize the waiting time associated with data access when the next artificial neural network operation is resumed when the processor is stopped in a state where the artificial neural network operation is not completed for whatever reason.

[0025] An object of the present disclosure is to provide a data management device that predicts in advance the storage address of data required for the next artificial neural network operation based on the type and characteristics of the artificial neural network operation, and responds in advance to the data request required for the next artificial neural network operation.

[0026] Conventionally, computation or data operations are performed only by data transfer between a single processor and a single memory device, but due to recent technological developments, computation or operations based on shared processors or shared memories are being used. At this time, cache coherence between local caches in each processor or memory is becoming an important issue. In data-intensive processing such as artificial neural network operations, this cache coherence is a more important issue.

[0027] One object of the present disclosure is not limited to a specific artificial neural network, but corresponds to various types or features of various artificial neural networks, and provides cache coherence between at least one processor and at least one memory in data operations of the artificial neural network. In addition, the object of the present disclosure is to provide a data management device, a data management method, and a computing system using the method, which are configured so that the configuration for artificial neural network data operations does not reduce the performance of general data operations other than artificial neural network data operations.

[0028] One object of the present disclosure is to minimize the latency associated with data access when resuming artificial neural network operation by using an apparatus implemented to address attempts to improve performance through multiple combinations of various types of artificial neural networks or dynamically deformable networks.

[0029] In addition, an object of the present disclosure is to optimize a data management device or a data cache memory in response to various types or characteristics of an artificial neural network, and proposes a data management device and a method for supporting an artificial neural network, which are capable of providing artificial neural network data operations, wherein the data cache storage is optimized for the artificial neural network operations at runtime.

[0030] An object of the present disclosure is to propose a data management device and method based on a reconfigurable memory, which can effectively support artificial neural network operations by optimizing hardware settings of a data cache memory corresponding to the type or characteristics of the artificial neural network. Means of solving the problem

[0031] The present disclosure is made as a means to solve the problems of the prior art, and a data management device according to an embodiment of the present disclosure includes: a processor interface circuit, which is used to receive an artificial neural network data operation request associated with an artificial neural network operation performed by at least one processor; a memory interface circuit, which is used to send a memory data operation request based on the artificial neural network data operation request to at least one memory, and receive memory data from the at least one memory in response to the memory data operation request; and an internal controller, which is used to generate the memory data operation request based on the artificial neural network data operation request.

[0032] The internal controller identifies the domain and type of the artificial neural network data operation request based on the identification information contained in the artificial neural network data operation request, and the internal controller generates data and address sequence information, which is the sequence of data of the artificial neural network data operation request associated with the identification information and the address of the data of the artificial neural network data operation request stored in the at least one memory in this sequence, and stores the data and address sequence information in the internal memory.

[0033] When a first artificial neural network data operation request is received, the internal controller generates information of a second address of data of a second artificial neural network data operation request following the first artificial neural network data operation request in the at least one memory as next address information based on information of a first address of data of the first artificial neural network data operation request stored in the at least one memory and the data and address sequence information stored in the internal memory.

[0034] The identification information may include information about whether the artificial neural network data operation request is a request for input data for training, a request for a weight assigned to each synapse of the artificial neural network, or a request for an activation parameter, and whether the currently received data request corresponds to the artificial neural network data operation request, the activation parameter being an operation result output from one layer of the artificial neural network to an output node of a next layer.

[0035] After receiving the first artificial neural network data operation request, the internal controller may predict the second artificial neural network data operation request that is expected to follow the first artificial neural network data operation request based on the first identification information included in the first artificial neural network data operation request.

[0036] In the absence of a previous artificial neural network data operation request, after receiving the first artificial neural network data operation request, the internal controller can generate a first memory data operation request including the first address information, and transmit the first memory data operation request to the memory interface circuit.

[0037] After receiving the first artificial neural network data operation request, the internal controller may generate a second memory data operation request including the next address information based on the first artificial neural network data operation, and transmit the second memory data operation request to the memory interface circuit.

[0038] At least one of the internal controller or the memory interface circuit may send the second memory data operation request to the at least one memory, control the at least one memory so that the at least one memory performs a memory operation corresponding to the second memory data operation request, and receive data corresponding to the second memory data operation request from the at least one memory when the second memory data operation request is a read operation.

[0039] At least one of the internal controller or the memory interface circuit may send the second memory data operation request to the at least one memory, and control the at least one memory so that the at least one memory maintains a ready state capable of performing a memory operation corresponding to the second memory data operation request.

[0040] When a third artificial neural network data operation request following the first artificial neural network data operation request is actually received from the at least one processor, the internal controller can check whether the third address information used to store the data in the third artificial neural network data operation request in the at least one memory is consistent with the next address information previously generated in the internal controller.

[0041] In this case, when the third address information is inconsistent with the next address information previously generated in the internal controller, the memory interface circuit may send a third memory data operation request including the third address information to the at least one memory, and the internal controller may update the data address sequence information so that the third address information of the third artificial neural network data operation request follows the first address information of the first artificial neural network data operation request, and may store the updated data address sequence information in the internal memory.

[0042] When using the data and address sequence information stored in the internal memory, the internal controller can determine the adaptive sequence information for generating the next address information based on the structure and characteristic information of the artificial neural network associated with the artificial neural network operation, whether the first artificial neural network data operation request is a read request or a write request, and the first identification information included in the first artificial neural network data operation request, and the first identification information included in the first artificial neural network data operation request, and the internal controller can predict the second artificial neural network data operation request based on the adaptive sequence information and the data-address sequence information stored in the internal memory, and generate the next address information. For example, when the first / second artificial neural network data operation request is associated with an operation of generating and storing activation parameters, the next address information can be generated in a raster scan order. Alternatively, when the first / second artificial neural network data operation request is associated with an operation of reading the weight of a synapse connected to a node of a next layer, it can be read according to the shape of a weight kernel. In this way, the next address information can be generated based on the adaptive sequence information according to the type of the artificial neural network data operation request and the related artificial neural network operation.

[0043] Based on the first identification information included in the first artificial neural network data operation request and whether the first artificial neural network data operation request is a read request or a write request, the internal controller can use the data and address sequence information stored in the internal memory to generate information of a current address of the data of the first artificial neural network data operation request stored in the at least one memory as the first address information, even if the first artificial neural network data operation request does not include an address.

[0044] The data management device may further include: a read data buffer memory, which, when the first artificial neural network data operation request is a read request, performs a pre-read operation on data predicted as the second artificial neural network data operation request from the at least one memory based on the next address information, and then stores pre-read data as a result of the pre-read operation.

[0045] A computing system according to an embodiment of the present disclosure is a computing system having at least one processor and at least one memory, and supporting an artificial neural network operation performed by the at least one processor, the computing system comprising: a processor-side interface for receiving an artificial neural network data operation request associated with the artificial neural network operation performed by the at least one processor; a memory-side interface for sending a memory data operation request based on the artificial neural network data operation request to the at least one memory, and receiving memory data from the at least one memory in response to the memory data operation request; and a data sequence memory for storing data and address sequence information.

[0046] The computing system generates the memory data operation request based on the artificial neural network data operation request, identifies the domain and type of the artificial neural network data operation request based on the identification information contained in the artificial neural network data operation request, generates the order of the data of the artificial neural network data operation request associated with the identification information and the information of the address of the data of the artificial neural network data operation request stored in the at least one memory in this order as the data and address sequence information, and, when the processor-side interface receives a first artificial neural network data operation request, the computing system generates the information of the second address of the data of the second artificial neural network data operation request following the first artificial neural network data operation request stored in the at least one memory as the next address information based on the information of the first address of the data of the first artificial neural network data operation request stored in the at least one memory and the data and address sequence information stored in the data sequence memory.

[0047] The memory-side interface may send the memory data operation request to the at least one memory or at least one external memory outside the computing system, and receive memory data from the at least one memory or the at least one external memory in response to the memory data operation request.

[0048] A data management method for supporting artificial neural network operations according to an embodiment of the present disclosure is a data management method performed in a computing system, the computing system having at least one processor and at least one memory, and supporting artificial neural network operations performed by the at least one processor, the data management method comprising: receiving an artificial neural network data operation request associated with the artificial neural network operation performed by the at least one processor; sending a memory data operation request based on the artificial neural network data operation request to the at least one memory, and receiving memory data from the at least one memory in response to the memory data operation request; and generating the memory data operation request based on the artificial neural network data operation request.

[0049] The method of generating the memory data operation request based on the artificial neural network data operation request includes: identifying the domain and type of the artificial neural network data operation request based on the identification information contained in the artificial neural network data operation request; generating information about the order of data of the artificial neural network data operation request associated with the identification information and the addresses of the data of the artificial neural network data operation request stored in the at least one memory in this order as the data and address sequence information; storing the data and address sequence information in a data sequence memory inside the computing system; and when a first artificial neural network data operation request is received, generating information about a second address of data of a second artificial neural network data operation request following the first artificial neural network data operation request in the at least one memory as next address information based on information about a first address of the data of the first artificial neural network data operation request stored in the at least one memory and the data and address sequence information stored in the data sequence memory.

[0050] A data cache device according to an embodiment of the present disclosure is a data cache device for caching data between at least one processor and at least one memory, the data cache device is a data cache device supporting artificial neural network operations performed by the at least one processor, and the data cache device may include: an internal controller that predicts a next data operation request based on artificial neural network data locality (ANN data locality) of the artificial neural network operations.

[0051] The internal controller of the data cache device may communicate with the at least one memory to pre-execute the next data operation request, and the data cache device further comprises: a cache memory for storing result data of the next data operation request executed by the at least one memory.

[0052] The internal controller of the data cache device may predict the next data operation request based on a current data operation request received from the at least one processor and the artificial neural network data locality.

[0053] The internal controller may extract the artificial neural network data locality by monitoring data operation requests associated with the artificial neural network operation among data operation requests actually made between the at least one processor and the at least one memory.

[0054] When the at least one processor sends a resume data operation request to the at least one memory in order to resume the artificial neural network data operation after the continuous artificial neural network operation between the at least one processor and the at least one memory is stopped, the internal controller can predict the next data operation request corresponding to the resume data operation request based on the continuous artificial neural network operation before being stopped and the artificial neural network data locality before receiving the resume data operation request, and provide the data operation for the next data operation request to the at least one processor without waiting time.

[0055] A data management device according to an embodiment of the present disclosure is a data management device for supporting artificial neural network operations, the data management device comprising: at least one processor and a reconfigurable memory, wherein, through program instructions executed in the at least one processor, the at least one processor analyzes a process of executing an artificial neural network operation and extracts a sequential pattern of artificial neural network data operations related to the artificial neural network operation, wherein, through the program instructions, the at least one processor generates artificial neural network memory configuration information based on the sequential pattern of the artificial neural network data operations; wherein, through the program instructions, the at least one processor controls the reconfigurable memory so as to apply the artificial neural network memory configuration information to the reconfigurable memory.

[0056] The at least one processor may extract the sequential pattern based on the artificial neural network data locality (ANN data locality) of the artificial neural network operation.

[0057] The at least one processor may extract the sequential pattern based on whether the artificial neural network data operation request is a request for input data for training, a request for a weight assigned to each synapse of the artificial neural network, or a request for an activation parameter, and whether a currently received data request corresponds to the artificial neural network data operation request, the activation parameter being a result of an operation from an output node of one layer of the artificial neural network to an output node of a next layer.

[0058] The at least one processor may extract a runtime sequential pattern of the artificial neural network data operation based on structure and property information of the artificial neural network associated with the artificial neural network operation, the sequential pattern, and whether the first artificial neural network data operation request is a read request or a write request.

[0059] A data management device for supporting artificial neural network operations according to an embodiment of the present disclosure includes: a controller and a reconfigurable memory, wherein, through program instructions executed in the controller, the controller receives a sequential pattern of artificial neural network data operations related to the artificial neural network operations obtained based on a process of executing the artificial neural network operations, wherein, through the program instructions, the controller generates artificial neural network memory configuration information based on the sequential pattern of the artificial neural network data operations; wherein, through the program instructions, the controller controls the reconfigurable memory so as to apply the artificial neural network memory configuration information to the reconfigurable memory.

[0060] The controller may generate the artificial neural network memory configuration information based on physical configuration information of the reconfigurable memory and the sequential pattern.

[0061] The controller may extract a runtime sequence pattern of the artificial neural network data operation based on the structure and characteristic information of the artificial neural network associated with the artificial neural network operation, the sequence pattern, and whether the first artificial neural network data operation request is a read request or a write request. In addition, when receiving the artificial neural network operation, the controller may cooperate with the reconfigurable memory to provide the artificial neural network data operation as a response to the artificial neural network operation based on the runtime sequence pattern.

[0062] A data management method for supporting artificial neural network operations according to an embodiment of the present disclosure is a data management method executed by program instructions executed in at least one processor of a data management device including at least one processor and a reconfigurable memory, the data management method comprising: according to the program instructions, analyzing, by the at least one processor, a process of executing an artificial neural network operation and extracting a sequential pattern of artificial neural network data operations related to the artificial neural network operation; according to the program instructions, generating, by the at least one processor, artificial neural network memory configuration information based on the sequential pattern of the artificial neural network data operations; and according to the program instructions, controlling, by the at least one processor, the reconfigurable memory so as to apply the artificial neural network memory configuration information to the reconfigurable memory.

[0063] A data management method for supporting artificial neural network operations according to an embodiment of the present disclosure is a data management method executed by program instructions executed in a controller of a data management device, the data management device including the controller and a reconfigurable memory and supporting artificial neural network operations, the data management method comprising: according to the program instructions, the controller receiving a sequential pattern of artificial neural network data operations related to the artificial neural network operations obtained based on a process of executing the artificial neural network operations; according to the program instructions, the controller generating artificial neural network memory configuration information based on the sequential pattern of the artificial neural network data operations; and according to the program instructions, the controller controlling the reconfigurable memory so as to apply the artificial neural network memory configuration information to the reconfigurable memory. Effects of the present disclosure

[0064] According to the present disclosure, even when an existing processor and an existing memory are used, in the case of resuming an artificial neural network operation that has been interrupted, the waiting time that occurs when the processor reads data from the memory can be removed or minimized.

[0065] According to the present disclosure, a delay time and an operation idle period for a processor to obtain data for an artificial neural network operation can be minimized, and a bandwidth of an existing memory can be maximized.

[0066] According to the present disclosure, the overall operational performance of an artificial neural network computing system may be optimized / maximized.

[0067] According to the present disclosure, a data management device and method are provided, which can support high-speed artificial neural network operations at any location between at least one processor and at least one memory device. In addition, a computing system in which such a data management method is executed can be provided.

[0068] According to the present disclosure, without being limited to a specific artificial neural network, cache coherence corresponding to various types or features of various artificial neural networks can be provided in artificial neural network data operations between at least one processor and at least one memory. In addition, a data management device, a data management method, and a computing system using the method can be provided, which are configured so that the configuration for artificial neural network data operations does not reduce the performance of general data operations other than artificial neural network data operations.

[0069] According to the present disclosure, latency associated with data access when resuming artificial neural network operations is minimized by using an apparatus implemented to address attempts to improve performance through multiple combinations of various types of artificial neural networks or dynamically deformable networks.

[0070] According to the present disclosure, a data management device or a data cache memory may be optimized in response to various types or characteristics of an artificial neural network, and an artificial neural network data operation may be provided in which a data cache memory is optimized for the artificial neural network operation at runtime.

[0071] According to the present disclosure, a reconfigurable memory can be provided which can effectively support an artificial neural network operation by optimizing a hardware setting of a data cache memory corresponding to the type or characteristics of the artificial neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 is a diagram showing the structure and operation of an example of a general artificial neural network.

[0073] Figure 2 is a diagram showing a data management device for supporting high-speed artificial neural network operations according to an embodiment of the present disclosure.

[0074] Figure 3 is a diagram showing a data management device for supporting high-speed artificial neural network operations according to another embodiment of the present disclosure.

[0075] Figure 4 is a diagram showing a situation in which a data management method for supporting an artificial neural network operation is executed in a computing system according to yet another embodiment of the present disclosure.

[0076] Figure 5is a diagram illustrating an example of data operations between a processor and a memory performed via a data management device according to an embodiment of the present disclosure.

[0077] Figure 6 is a diagram illustrating another example of a data operation between a processor and a memory performed via a data management device according to an embodiment of the present disclosure.

[0078] Figure 7 is an operational flow chart showing some processes of a data management method according to an embodiment of the present disclosure.

[0079] Figure 8 is an operational flow chart showing some processes of a data management method according to an embodiment of the present disclosure.

[0080] Fig. 9 is an operational flow chart showing some processes of a data management method according to an embodiment of the present disclosure.

[0081] Fig.10 is a diagram showing data and address sequence information according to an embodiment of the present disclosure.

[0082] Fig.11 is a diagram showing data and address sequence information according to an embodiment of the present disclosure.

[0083] Fig.12 is a diagram showing data and address sequence information according to an embodiment of the present disclosure.

[0084] Figure 1 3 is a diagram showing a method of expressing channel information / identification information according to various embodiments of the present disclosure.

[0085] Fig.14 is a diagram showing a case in which channel information / identification information is associated with data and address sequence information according to an embodiment of the present disclosure.

[0086] Fig.15 is a diagram illustrating a data management device that is adaptively reconfigured to support artificial neural network operations according to an embodiment of the present disclosure.

[0087] Fig.16 is a diagram illustrating a data management device adaptively reconfigured to support artificial neural network operations according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0088] The present disclosure is made as a means to solve the problems of the prior art, and a data management device according to an embodiment of the present disclosure includes: a processor interface circuit, which is used to receive an artificial neural network data operation request associated with an artificial neural network operation performed by at least one processor; a memory interface circuit, which is used to send a memory data operation request based on the artificial neural network data operation request to at least one memory, and receive memory data from the at least one memory in response to the memory data operation request; and an internal controller, which is used to generate the memory data operation request based on the artificial neural network data operation request.

[0089] The internal controller identifies the domain and type of the artificial neural network data operation request based on the identification information contained in the artificial neural network data operation request, and the internal controller generates data and address sequence information, which is the sequence of data of the artificial neural network data operation request associated with the identification information and the address of the data of the artificial neural network data operation request stored in the at least one memory in this sequence, and stores the data and address sequence information in the internal memory.

[0090] When a first artificial neural network data operation request is received, the internal controller generates information of a second address of data of a second artificial neural network data operation request following the first artificial neural network data operation request in the at least one memory as next address information based on information of a first address of data of the first artificial neural network data operation request stored in the at least one memory and the data and address sequence information stored in the internal memory.

[0091] The identification information may include information about whether the artificial neural network data operation request is a request for input data for training, a request for a weight assigned to each synapse of the artificial neural network, or a request for an activation parameter, and whether the currently received data request corresponds to the artificial neural network data operation request, the activation parameter being an operation result output from one layer of the artificial neural network to an output node of a next layer.

[0092] After receiving the first artificial neural network data operation request, the internal controller may predict the second artificial neural network data operation request that is expected to follow the first artificial neural network data operation request based on the first identification information included in the first artificial neural network data operation request.

[0093] In the absence of a previous artificial neural network data operation request, after receiving the first artificial neural network data operation request, the internal controller can generate a first memory data operation request including the first address information, and transmit the first memory data operation request to the memory interface circuit.

[0094] After receiving the first artificial neural network data operation request, the internal controller may generate a second memory data operation request including the next address information based on the first artificial neural network data operation, and transmit the second memory data operation request to the memory interface circuit.

[0095] At least one of the internal controller or the memory interface circuit may send the second memory data operation request to the at least one memory, control the at least one memory so that the at least one memory performs a memory operation corresponding to the second memory data operation request, and receive data corresponding to the second memory data operation request from the at least one memory when the second memory data operation request is a read operation.

[0096] At least one of the internal controller or the memory interface circuit may send the second memory data operation request to the at least one memory, and control the at least one memory so that the at least one memory maintains a ready state capable of performing a memory operation corresponding to the second memory data operation request.

[0097] When a third artificial neural network data operation request following the first artificial neural network data operation request is actually received from the at least one processor, the internal controller can check whether the third address information used to store the data in the third artificial neural network data operation request in the at least one memory is consistent with the next address information previously generated in the internal controller.

[0098] In this case, when the third address information is inconsistent with the next address information previously generated in the internal controller, the memory interface circuit may send a third memory data operation request including the third address information to the at least one memory, and the internal controller may update the data address sequence information so that the third address information of the third artificial neural network data operation request follows the first address information of the first artificial neural network data operation request, and may store the updated data address sequence information in the internal memory.

[0099] When using the data and address sequence information stored in the internal memory, the internal controller can determine the adaptive sequence information for generating the next address information based on the structure and characteristic information of the artificial neural network associated with the artificial neural network operation, whether the first artificial neural network data operation request is a read request or a write request, and the first identification information included in the first artificial neural network data operation request, and the first identification information included in the first artificial neural network data operation request, and the internal controller can predict the second artificial neural network data operation request based on the adaptive sequence information and the data-address sequence information stored in the internal memory, and generate the next address information. For example, when the first / second artificial neural network data operation request is associated with an operation of generating and storing activation parameters, the next address information can be generated in a raster scan order. Alternatively, when the first / second artificial neural network data operation request is associated with an operation of reading the weight of a synapse connected to a node of a next layer, it can be read according to the shape of a weight kernel. In this way, the next address information can be generated based on the adaptive sequence information according to the type of the artificial neural network data operation request and the related artificial neural network operation.

[0100] Based on the first identification information included in the first artificial neural network data operation request and whether the first artificial neural network data operation request is a read request or a write request, the internal controller can use the data and address sequence information stored in the internal memory to generate information of a current address of the data of the first artificial neural network data operation request stored in the at least one memory as the first address information, even if the first artificial neural network data operation request does not include an address.

[0101] The data management device of the present disclosure may further include: a read data buffer memory, which, when the first artificial neural network data operation request is a read request, performs a pre-read operation on data predicted as the second artificial neural network data operation request from the at least one memory based on the next address information, and then stores the pre-read data as a result of the pre-read operation.

[0102] A computing system having at least one processor and at least one memory, and supporting an artificial neural network operation performed by the at least one processor, the computing system comprising: a processor-side interface for receiving an artificial neural network data operation request associated with the artificial neural network operation performed by the at least one processor; a memory-side interface for sending a memory data operation request based on the artificial neural network data operation request to the at least one memory, and receiving memory data from the at least one memory in response to the memory data operation request; and a data sequence memory for storing data and address sequence information.

[0103] The computing system generates the memory data operation request based on the artificial neural network data operation request, identifies the domain and type of the artificial neural network data operation request based on the identification information contained in the artificial neural network data operation request, generates the order of the data of the artificial neural network data operation request associated with the identification information and the information of the address of the data of the artificial neural network data operation request stored in the at least one memory in this order as the data and address sequence information, and, when the processor-side interface receives a first artificial neural network data operation request, the computing system generates the information of the second address of the data of the second artificial neural network data operation request following the first artificial neural network data operation request stored in the at least one memory as the next address information based on the information of the first address of the data of the first artificial neural network data operation request stored in the at least one memory and the data and address sequence information stored in the data sequence memory.

[0104] The memory-side interface may send the memory data operation request to the at least one memory or at least one external memory outside the computing system, and receive memory data from the at least one memory or the at least one external memory in response to the memory data operation request.

[0105] A data management method for supporting artificial neural network operations according to an embodiment of the present disclosure is a data management method performed in a computing system, the computing system having at least one processor and at least one memory, and supporting artificial neural network operations performed by the at least one processor, the data management method comprising: receiving an artificial neural network data operation request associated with the artificial neural network operation performed by the at least one processor; sending a memory data operation request based on the artificial neural network data operation request to the at least one memory, and receiving memory data from the at least one memory in response to the memory data operation request; and generating the memory data operation request based on the artificial neural network data operation request.

[0106] The method of generating the memory data operation request based on the artificial neural network data operation request includes: identifying the domain and type of the artificial neural network data operation request based on the identification information contained in the artificial neural network data operation request; generating information about the order of data of the artificial neural network data operation request associated with the identification information and the addresses of the data of the artificial neural network data operation request stored in the at least one memory in this order as the data and address sequence information; storing the data and address sequence information in a data sequence memory inside the computing system; and when a first artificial neural network data operation request is received, generating information about a second address of data of a second artificial neural network data operation request following the first artificial neural network data operation request in the at least one memory as next address information based on information about a first address of the data of the first artificial neural network data operation request stored in the at least one memory and the data and address sequence information stored in the data sequence memory.

[0107] A data cache device according to an embodiment of the present disclosure is a data cache device for caching data between at least one processor and at least one memory, the data cache device is a data cache device supporting artificial neural network operations performed by the at least one processor, and the data cache device may include: an internal controller that predicts a next data operation request based on artificial neural network data locality (ANN data locality) of the artificial neural network operations.

[0108] The internal controller of the data cache device may communicate with the at least one memory to pre-execute the next data operation request, and the data cache device further comprises: a cache memory for storing result data of the next data operation request executed by the at least one memory.

[0109] The internal controller of the data cache device may predict the next data operation request based on a current data operation request received from the at least one processor and the artificial neural network data locality.

[0110] The internal controller may extract the artificial neural network data locality by monitoring data operation requests associated with the artificial neural network operation among data operation requests actually made between the at least one processor and the at least one memory.

[0111] When the at least one processor sends a resume data operation request to the at least one memory in order to resume the artificial neural network data operation after the continuous artificial neural network operation between the at least one processor and the at least one memory is stopped, the internal controller can predict the next data operation request corresponding to the resume data operation request based on the continuous artificial neural network operation before being stopped and the artificial neural network data locality before receiving the resume data operation request, and provide the data operation for the next data operation request to the at least one processor without waiting time.

[0112] A data management device according to an embodiment of the present disclosure is a data management device for supporting artificial neural network operations, the data management device comprising: at least one processor and a reconfigurable memory, wherein, through program instructions executed in the at least one processor, the at least one processor analyzes a process of executing an artificial neural network operation and extracts a sequential pattern of artificial neural network data operations related to the artificial neural network operation, wherein, through the program instructions, the at least one processor generates artificial neural network memory configuration information based on the sequential pattern of the artificial neural network data operations; wherein, through the program instructions, the at least one processor controls the reconfigurable memory so as to apply the artificial neural network memory configuration information to the reconfigurable memory.

[0113] The at least one processor may extract the sequential pattern based on the artificial neural network data locality (ANN data locality) of the artificial neural network operation.

[0114] The at least one processor may extract the sequential pattern based on whether the artificial neural network data operation request is a request for input data for training, a request for a weight assigned to each synapse of the artificial neural network, or a request for an activation parameter, and whether a currently received data request corresponds to the artificial neural network data operation request, the activation parameter being a result of an operation from an output node of one layer of the artificial neural network to an output node of a next layer.

[0115] The at least one processor may extract a runtime sequential pattern of the artificial neural network data operation based on structure and property information of the artificial neural network associated with the artificial neural network operation, the sequential pattern, and whether the first artificial neural network data operation request is a read request or a write request.

[0116] According to an embodiment of the present disclosure, a data management device for supporting artificial neural network operations includes: a controller and a reconfigurable memory, wherein, through program instructions executed in the controller, the controller receives a sequential pattern of artificial neural network data operations related to the artificial neural network operations obtained based on a process of executing the artificial neural network operations, wherein, through the program instructions, the controller generates artificial neural network memory configuration information based on the sequential pattern of the artificial neural network data operations; wherein, through the program instructions, the controller controls the reconfigurable memory so as to apply the artificial neural network memory configuration information to the reconfigurable memory.

[0117] The controller may generate the artificial neural network memory configuration information based on physical configuration information of the reconfigurable memory and the sequential pattern.

[0118] The controller may extract a runtime sequence pattern of the artificial neural network data operation based on the structure and characteristic information of the artificial neural network associated with the artificial neural network operation, the sequence pattern, and whether the first artificial neural network data operation request is a read request or a write request. In addition, when receiving the artificial neural network operation, the controller may cooperate with the reconfigurable memory to provide the artificial neural network data operation as a response to the artificial neural network operation based on the runtime sequence pattern.

[0119] A data management method for supporting artificial neural network operations according to an embodiment of the present disclosure is a data management method executed by program instructions executed in at least one processor of a data management device including at least one processor and a reconfigurable memory, the data management method comprising: according to the program instructions, analyzing, by the at least one processor, a process of executing an artificial neural network operation and extracting a sequential pattern of artificial neural network data operations related to the artificial neural network operation; according to the program instructions, generating, by the at least one processor, artificial neural network memory configuration information based on the sequential pattern of the artificial neural network data operations; and according to the program instructions, controlling, by the at least one processor, the reconfigurable memory so as to apply the artificial neural network memory configuration information to the reconfigurable memory.

[0120] A data management method for supporting artificial neural network operations according to an embodiment of the present disclosure is a data management method executed by program instructions executed in a controller of a data management device, the data management device including the controller and a reconfigurable memory and supporting artificial neural network operations, the data management method comprising: according to the program instructions, the controller receiving a sequential pattern of artificial neural network data operations related to the artificial neural network operations obtained based on a process of executing the artificial neural network operations; according to the program instructions, the controller generating artificial neural network memory configuration information based on the sequential pattern of the artificial neural network data operations; and according to the program instructions, the controller controlling the reconfigurable memory so as to apply the artificial neural network memory configuration information to the reconfigurable memory. Specific implementation plan

[0121] In addition to the above-mentioned objects, other objects and features of the present disclosure will become apparent through the description of the embodiments with reference to the accompanying drawings.

[0122] Here, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the following description of the present disclosure, when a detailed description of known functions and configurations incorporated herein may make the subject matter of the present disclosure unclear, it will be omitted.

[0123] Figure 1 is a diagram showing the structure and operation of an example of a general artificial neural network. Figure 1 Part or all of the configuration of the artificial neural network may be included as part of the present disclosure for a specific implementation of the present disclosure.

[0124] In addition, a method may be included as a part of the present disclosure for a specific implementation of the present disclosure, in which the processor Figure 1 The artificial neural network accesses memory during the data operations associated with it.

[0125] refer to Figure 1 , an example of a convolutional neural network (CNN) including a hidden layer is shown. A plurality of input values ​​10 are sent to a plurality of input nodes 11. The values ​​of the corresponding input nodes 11 are multiplied by the corresponding weights 14 and sent to the nodes of the next layer. For example, the weighted sum of the values ​​output from the input nodes 11 may be normalized and sent to the hidden nodes 12 of the hidden layer. The values ​​calculated by the plurality of hidden layers are sent to the hidden nodes 16 of the last hidden layer. The values ​​of the nodes 16 of the last hidden layer are multiplied by the weights 14 and transmitted to the output nodes 13, which ultimately provide the output values ​​15 as the final result of the artificial neural network.

[0126] In various fields such as pattern analysis in images, comparison between images, and pattern analysis in audio / text, the rapid development of artificial neural network technology based on deep learning / CNN in recent years has led to performance superior to existing rule-based pattern recognition technology. Artificial neural network technology that recognizes and detects patterns that are difficult for the human eye and ear to distinguish is expected to expand into various fields such as security, medical imaging, and non-destructive testing.

[0127] When the artificial neural network is trained based on the input value 10, the weight 14 representing the relationship between the data corresponding to the nodes 11, 12, 13, 16 of the corresponding layer and the corresponding nodes is calculated by Figure 1 At this time, during the training process, the processor reads the weights 14 and the data corresponding to the nodes 11, 12, 13, 16 of the corresponding layer from the memory, and overwrites the data changed by the operation in the memory.

[0128] The data required in the training process of artificial neural network is classified into input data for training, weight data (related parameters) and activation parameters (feature map). The input data for training can be images, audio, vibration data, etc. For example, in the case of images, the pixel values ​​are input to the artificial neural network as Figure 1 The input values ​​10 are stored in the memory once during the training process of the artificial neural network, and then read repeatedly tens to hundreds of times and used in the training process. The input values ​​10 are read tens to hundreds of times in the order in which they were originally stored in the memory.

[0129] Weight data is a combination of weight values ​​assigned to each synapse of the artificial neural network. Figure 1As shown in weight 14, it is a relevant parameter indicating the relationship between the corresponding nodes. During the training process of the artificial neural network, the weight data should be read from the memory to upgrade from one layer to the next layer, and in some cases, the weight data should be modified, and the modified weight data should be overwritten in the memory. The training process of the artificial neural network is a process of repeatedly performing necessary operations by reading a series of learning data fragments and total weight data contained in the artificial neural network model, and thereby generating a modified weight value. The training process is performed on all learning data, and the weight is modified by repeating the operation on a set of learning data dozens to hundreds of times. The order of overwriting weight values ​​during the training process of a series of learning data is the order in which the weight values ​​are initially stored in the write operation of the weight values, and the order of reading weight values ​​is based on the order in which the weight values ​​are initially written in the write operation or the order in which the weight values ​​are modified in the previous training process.

[0130] The activation parameter is the calculation result data that is output from one layer of the artificial neural network to the nodes of the next layer. In order to calculate the next layer of the corresponding layer, the feature map data of the corresponding layer is read and the value of the next layer is calculated. If such activation parameters / feature maps are stored in the memory in the order in which they were created and read for the next layer operation, the reading order will be determined based at least on the order in which they are stored in the memory. In most cases, the activation parameters / feature maps are generated in one layer, and their effect ends when they are input to the next layer. Figure 1 In the example, when the value of node 11 is multiplied by weight 14 and transmitted to node 12, the value of node 11 is usually rarely used in the subsequent process, and it can be seen that its role ends. However, depending on the application and purpose, there is a case where changes in the values ​​of nodes of each layer are monitored (in the case of search operations inside artificial neural networks). In this case, the values ​​of nodes of each layer are also stored in the memory.

[0131] When the training of the artificial neural network is terminated, if a new input value 10 is given, the artificial neural network infers the new input value 10 as a result of the training by applying the fixed weight 14 and provides an output value 15.

[0132] During the inference process of the artificial neural network, new input values ​​10 are read from the memory, and weights 14 are also sequentially read from the memory. Unlike the training process, since the weights 14 do not change during the inference process, there is no need for a weight 14 overwriting process. The activation parameters of each node usually end their role after the operation of their corresponding layer and are rarely used thereafter.

[0133] The feature map data of each node usually ends its role after the operation of its corresponding layer, but recently, attempts have been made to analyze the actions within the artificial neural network. Depending on the application and purpose, there is a case where the feature map data of each node is stored in the memory. In this case, the feature map data of each node can be stored in the memory according to the request of the processor.

[0134] In the reasoning process of the artificial neural network, since the weight value is fixed, there is no need for an overwrite operation, and the data originally stored in the memory is read as is. At this time, the order of the read operation will be determined based on the order of the write operation, or the order of the read operation can be applied by partially changing the order of the write operation, and the order of the write operation is equally applied to the next reasoning process.

[0135] For activation parameters / feature maps, data is generated by a layer and stored in memory during inference, and its role in the inference process ends when it is used as input to the next layer. In most cases, activation parameters / feature maps are read in the order in which the data pieces are first stored in memory during inference.

[0136] Despite Figure 1 The convolutional neural network is mainly shown in the figure, but the present disclosure is not limited to the convolutional neural network and can be applied to the case of using any artificial neural network (ANN) other than the convolutional neural network.

[0137] The disclosures of the above-mentioned previous documents (e.g., KR 10-2019-018888, US 2019 / 0057302, US 9,805,304, US2019 / 0057300, US 9,836,691, etc.) may be included within the scope of the purpose of the present disclosure as part of the configuration of the present disclosure if necessary for implementation.

[0138] Figure 2 is a diagram illustrating a data management device 210 for supporting high-speed artificial neural network operations according to an embodiment of the present disclosure.

[0139] Figure 2At least one processor 260 shown in FIG. 1 performs an artificial neural network operation alone or in cooperation with another processor, and reads data associated with the artificial neural network operation from at least one memory 270, or stores data associated with the artificial neural network operation in at least one memory 270. In this case, a read or write request to at least one memory 270 as data associated with the artificial neural network operation will be referred to as an artificial neural network data operation request. The artificial neural network data operation request may refer to data associated with the artificial neural network operation requested by the processor 260 from the memory 270, or may refer to data that needs to be stored in the memory 270 by the processor 260. The artificial neural network data operation request includes information on at least one of a data field or an address field according to whether a read request or a write request is made.

[0140] At least one processor 260 can be implemented with any type of processor capable of performing artificial neural network calculations. For example, it can include all types of processors customized for artificial neural network calculations, as well as general-purpose CPUs, GPUs dedicated to graphics processing, and neural network processors (NNPs) dedicated to artificial neural network calculations.

[0141] The data management device 210 of the present disclosure includes an internal controller 220 , a processor-side interface circuit 240 , and a memory-side interface circuit 250 .

[0142] The processor-side interface circuit 240 receives an artificial neural network data operation request from at least one processor 260. The memory-side interface circuit 250 sends a memory data operation request based on the artificial neural network data operation request to at least one memory 270, and receives memory data corresponding to the memory data operation request from at least one memory 270.

[0143] The internal controller 220 generates a memory data operation request based on the artificial neural network data operation request.

[0144] The internal controller 220 may include an access data type detection submodule that identifies the domain and type of the artificial neural network data operation request based on the identification information / channel information included in the artificial neural network data operation request. At this time, the type of the artificial neural network data operation request can be distinguished based on the identification information / channel information. The type of the artificial neural network data operation request refers to the data type designated as the object of the data operation requested in the memory, and may correspond to any one of the input data, weight data, or activation parameters (feature map) in the artificial neural network. In addition, depending on whether the operation request is read or write, future predictions and responses may vary.

[0145] The internal controller 220 may monitor, track or observe a data sequence of artificial neural network data operation requests associated with the identification information / channel information, and information of addresses of data of the artificial neural network data operation requests stored in that sequence in at least one memory, and in this case, the internal controller 220 generates the monitored, tracked and observed address information as data and address sequence information, and stores the data and address sequence information in the memory 230 as the internal memory.

[0146] The data sequential memory 230 is a memory located inside the data management device 210 and is distinguished from at least one memory 270 located outside the data management device 210. Figure 2 2 as being located outside the internal controller 220 , but according to another embodiment, the data sequential memory 230 may be located inside the internal controller 220 .

[0147] When a first artificial neural network data operation request is received, the next read / write address generator submodule of the internal controller 220 may generate information of a second address of data of a second artificial neural network data operation request following the first artificial neural network data operation request in at least one memory 270 as next information based on information of a first address of data of the first artificial neural network data operation request stored in at least one memory 270 and the data and address sequence information stored in the data sequence memory 230 as the internal memory. When an Nth artificial neural network data operation request is received, the internal controller 220 may predict and generate information of an (N+1)th address of data of an (N+1)th artificial neural network data operation request following the Nth artificial neural network data operation request in at least one memory 270 based on information of an Nth address of data of the Nth artificial neural network data operation request stored in at least one memory 270 and the data and address sequence information stored in the data sequence memory 230 as the internal memory.

[0148] When the internal controller 220 predicts the second artificial neural network data operation request to be executed next and generates the next address information based on the first artificial neural network data operation request that has been currently received, the internal controller 220 may refer to the identification information / channel information included in the first artificial neural network data operation request. As described above, the internal controller 220 monitors, tracks or observes the order of data of the artificial neural network data operation request associated with the identification information / channel information and the information of the address of the data of the artificial neural network data operation request stored in the order in at least one memory, and at this time, the internal controller 220 may generate the monitored, tracked or observed address information as data and address sequence information. The internal controller 220 may predict the second artificial neural network data operation request to be executed next and generate the next address information based on the data and address sequence information obtained as a result of monitoring, tracking or observation and the first artificial neural network data operation request that has been currently received.

[0149] The identification information / channel information may include information about whether the artificial neural network data operation request is a request for input data for training, a request for a weight assigned to each synapse of the artificial neural network, or a request for an activation parameter, and whether the currently received data request corresponds to the artificial neural network data operation request, and the activation parameter is an operation result output from one layer of the artificial neural network to an output node of the next layer.

[0150] The identification information / channel information can be represented by an identification code / channel code. The domain of the artificial neural network data classified by the identification code can be classified into input data for training, weight data (association parameters), and activation parameters (feature maps).

[0151] Furthermore, the identification code may indicate whether the data operation request is an artificial neural network data operation by designating, as a separate state value, a case where the data operation request currently received from the processor 260 is not associated with an artificial neural network operation.

[0152] After receiving the first artificial neural network data operation request, the internal controller 220 may predict a second artificial neural network data operation request that is expected to follow the first artificial neural network data operation request based on the first identification information included in the first artificial neural network data operation. Artificial neural network data operation request. That is, the internal controller may predict the second artificial neural network data operation request to be executed next based on the type of the first artificial neural network data operation request. In this case, the prediction of the second artificial neural network data operation request may include a prediction of whether the request is for reading or writing, and may include a prediction of an address of a memory where data is to be stored or has been stored.

[0153] When the data of the write request is sequentially stored in the at least one memory 270 after the processor-side interface circuit 240 receives a write request for data required for the operation of the artificial neural network from the at least one processor 260 and transmits it to the at least one memory 270 through the memory-side interface circuit 250, the data and address sequence information stored in the data sequence memory 230 as the internal memory may be first generated (initialized). The data sequence of the write request and the information of the address of the data of the write request stored in the at least one memory 270 in accordance with the sequence are generated as the data and address sequence information. The simplest example may be a table in which the data of the write request and the address in the at least one memory 270 corresponding thereto are sequentially described. The data of the write request may be divided into units such as bytes, words, sectors, pages, and blocks, and the address in the at least one memory 270 corresponding to each divided unit may be sequentially stored in the table. The data sequence memory 230 as the internal memory may store the table.

[0154] If the address information of each piece of data about the write request is stored in the table in this way, the size of the table will be too large, so the table can be compressed as needed. That is, the continuous data of the write request is represented only by the start address and the end address, and the data between the two can be regarded as being stored sequentially between the start address and the end address.

[0155] When the at least one memory 270 is a non-volatile memory such as a flash memory, the data and address sequence information stored in the data sequence memory 230 may include at least a portion of information of a physical-logical address mapping table or a flash translation layer (FTL). When the configuration of the at least one memory 270 is changed, the at least one memory 270 transmits information about the changed configuration to the data management device 210, and the internal controller 220 of the data management device 210 may update the data and address sequence information based on the information about the changed configuration and store the updated data and address sequence information in the data sequence memory 230.

[0156] Even if the configuration of the artificial neural network recognized by the at least one processor 260 is changed, the internal controller 220 of the data management device 210 can still update the data and address sequence information required for the artificial neural network operation based on the changed configuration data of the artificial neural network. The address sequence information can be updated. For example, in the case of applying the dynamically changed artificial neural network technology, the at least one processor 260 starts the artificial neural network operation, assuming that the artificial neural network has 100 hidden layers, but the strategy can be changed to perform the artificial neural network operation using only 70 hidden layers. In this case, the at least one processor 260 can send the dynamically changed configuration of the artificial neural network to the data management device 210. However, in general, the at least one processor 260 can send the artificial neural network data operation request to the data management device 210 without notifying the dynamically changed configuration of the artificial neural network. In this case, the data management device 210 can calculate that the configuration of the artificial neural network has changed by monitoring the information of the address in the at least one memory 270 when the artificial neural network data operation request received from the at least one processor 260 is sequentially accessed. Even when the at least one processor 260 notifies or does not notify the dynamically changed configuration of the artificial neural network, the internal controller 220 of the data management device 210 can detect the change in the configuration of the artificial neural network and update the data and address sequence information stored in the data sequence memory 230.

[0157] In the process of processing a data write request associated with an artificial neural network operation, data and address sequence information is first generated (initialized), and in the process of at least one processor 260 requesting to read data stored in at least one memory 270, the internal controller 220 in the data management device 210 can be used to predict the data and address of the next artificial neural network data operation request.

[0158] For example, image data can be used as input data required for training an artificial neural network. Image data can be represented as brightness values ​​and colors of each pixel. When a write request is made, fragments of information of each pixel are sequentially stored in at least one memory 270, and there is a high probability that they are called from at least one memory 270 in the same order as the write request when a read request is made. In addition, when a write request is made, feature maps or activation parameters of each layer are also sequentially stored in at least one memory 270, and even when a read request is made, there is a high probability that they are called from at least one memory 270 in the same order as the write request.

[0159] At the same time, even in Figure 1In the case of weight 14 data displayed in the artificial neural network, when a read request is made, the data and address sequence information generated when the write request is made can be used. However, the order of weight 14 data when a write request is made may be different from the order of weight 14 data when a read request is made. Since there is the possibility that when a write request is made, the weight 14 data can be stored in the order of nodes of the previous layer, and when a read request is made, the weight 14 data can be called in the order of nodes of the next layer, the internal controller 220 inside the data management device 210 can predict the next data operation request by identifying the type and characteristics of the artificial neural network data operation request. Even in this case, the internal controller 220 can predict the next data operation request based on the data and address sequence information that has been previously stored in the data sequence memory 230. The internal controller 220 can distinguish whether the corresponding artificial neural network data operation request is read / write, training process / inference process, etc. by using the identification code / channel code included in the artificial neural network data operation request, and predict the next data operation request based on the data and address sequence information according to the distinguished type and characteristics of the artificial neural network data operation request.

[0160] Meanwhile, for data having different call orders when a write request is made and when a read request is made, such as weight 14 data, the address within the at least one memory 270 may be allocated in consideration of factors affecting the order from the time stored in the at least one memory 270. For example, at least a portion of the address in the at least one memory 270 may be generated by combining the order of the nodes of the upper layer associated with the weight and the order of the nodes of the lower layer. In this case, the internal controller 220 may generate and manage separate data and address order information by assuming the order when a write request is made and when a read request is made, respectively.

[0161] The internal controller 220 receives the first artificial neural network data operation request in a state where there is no previous artificial neural network data operation request, and then, may generate a first memory data operation request including first address information of the first artificial neural network data operation request, and send the first memory data operation request to the memory side interface circuit 250. That is, when a write request associated with an artificial neural network operation is initiated without previously stored data, the data management device 210 may send each write request (including data and address) to at least one memory 270. At this time, mapping information between data and address of the write request is first generated (initialized) as data and address sequence information, and the first generated (initialized) data and address sequence information is stored in the data sequence memory 230.

[0162] After receiving the first artificial neural network data operation request, the internal controller 220 may generate a second memory data operation request including next address information based on the first artificial neural network data operation, and send the second memory data operation request to the memory interface circuit. That is, in a state where data and address sequence information are first generated (initialized) by a write request, when a read request or an overwrite request associated with an artificial neural network operation for stored data is received, the internal controller 220 may predict the second artificial neural network data operation request based on the first artificial neural network data operation request that has been currently received based on the data and address sequence information, even before receiving the next second artificial neural network data operation request. As described above, such prediction may be achieved using the following features: artificial neural network data operations are generally data-intensive operations and use specific rules to access the next data.

[0163] In a state where the internal controller 220 receives sequential artificial neural network data operation requests from at least one processor 260 based on an artificial neural network operation, if the next artificial neural network data operation request is delayed or not received due to various reasons, the internal controller 220 may predict the next artificial neural network data operation request based on the artificial neural network data operation requests that have been received so far, and send the required actions thereof to at least one memory 270. At this time, the next artificial neural network data operation request predicted by the internal controller 220 is sent to at least one memory 270, and the internal controller 220 may select data and an address and send them to at least one memory 270. The at least one memory 270 may maintain a ready state to perform a memory operation according to the next artificial neural network data operation request predicted by the internal controller 220, or may output data by performing a memory operation. If the next read request is not received when the sequential read request is performed in association with the artificial neural network operation request, the next read request predicted by the internal controller 220 is sent to the at least one memory 270, and the at least one memory 270 may perform a memory operation based on the next read request even before the next read request arrives from the processor 260 to the data management device 210, and read the data of the next read request and provide it in advance to the data management device 270. Thereafter, when the processor 260 resumes the operation or removes the obstacle that delays the read request, and thus the data management device 270 receives the next read request from the processor 260, the data management device 210 may quickly send the data of the next read request provided in advance from the memory 270 to the processor 260 without waiting time or loss.

[0164] That is, if the time for which the read request from the processor 260 stops is longer than the normal delay time, the memory 270 is likely to enter another memory operation. Therefore, in the prior art, when the next read request is sent from the processor 260 later, the memory 270 needs to re-execute the preparation operation from it so as to execute the next read request after the read request stops and resumes. The delay or loss generated in this process may be a factor that reduces the speed of artificial neural network data operation. When the read request from the processor 260 is resumed after stopping, the data management device 210 according to the present disclosure can quickly provide the data of the next read request prepared in advance to the processor 260, so that it can make the waiting time or loss recognized by the processor 260 zero or minimize the waiting time or loss. Although the data management device 210 sends the data of the next read request provided in advance to the processor 260, it can send a sequential read request to the memory 270 so that the memory 270 can respond to the sequential read request following the next read request. Therefore, the memory 270 can perform subsequent memory operations without waiting time or loss. When receiving the first artificial neural network data operation request (A), the internal controller 220 not only sends it to the memory 270 so that the first artificial neural network data operation (A) is executed without interruption, but also predicts the second artificial neural network data operation request (B′) following the first artificial neural network data operation (A), transmits the predicted second artificial neural network data operation request (B′) to the memory 270, and then receives the result data of the second artificial neural network data operation request (B′) from the memory 270 in advance.

[0165] Thereafter, when a third artificial neural network data operation request (B) that actually follows the first artificial neural network data operation request (A) is received from the processor 260, if the third artificial neural network data operation request (B) and the predicted second artificial neural network data operation request (B′) are identical to each other, the data management device 210 sends the result data of the second artificial neural network data operation request (B′) previously received from the memory 270 to the processor 260, thereby minimizing the waiting time or loss. The internal controller 220 may predict the fourth artificial neural network data operation request (C′) to follow next based on the actually received third artificial neural network data operation request (B), send the fourth artificial neural network data operation request (C′) to the memory 270, and then receive the result data of the fourth artificial neural network data operation request (C′) from the memory 270 in advance.

[0166] Even if the data request associated with the artificial neural network operation request is stopped or delayed after the first artificial neural network data operation request (A), and the third artificial neural network data operation request (B) is resumed after a long period of time, the data management device 210 according to the present disclosure can send the result data of the second artificial neural network data operation request B' prepared in advance to the processor 260, thereby minimizing the waiting time or loss. In addition, the internal controller 220 can predict the fourth artificial neural network data operation request (C') to follow based on the third artificial neural network data operation request (B) actually received, send the fourth artificial neural network data operation request (c') to the memory 270, and then receive the result data of the fourth artificial neural network data operation request (C') from the memory 270 in advance, so that the memory 270 can be controlled to quickly execute the next data operation. This enables the resumed data operation to be performed quickly without causing waiting time or loss.

[0167] When the check legal access submodule of the internal controller 220 actually receives a third artificial neural network data operation request (B) following the first artificial neural network data operation request (A) from at least one processor 260, it can check whether the third address information used for storing the data of the artificial neural network data operation request (B) in at least one memory 270 is consistent with the address information of the second artificial neural network data operation request (B′) previously generated / predicted as the next address information in the internal controller 220.

[0168] At this time, if the third address information is inconsistent with the address information of the second artificial neural network data operation request (B′) previously generated / predicted as the next address information in the internal controller 220, the memory side interface circuit 250 can send a third memory data operation request (B) including the third address information to at least one memory 270, and the internal controller 220 can update the data and address sequence information so that the third address information of the third artificial neural network data operation request B follows the first address information of the first artificial neural network data operation request (A), and stores the updated data and address sequence information in the data sequence memory 230 serving as the internal memory.

[0169] In this way, when the processor 260 performs a read access in an unexpected order, since the data management device 210 cannot prepare the data of the third artificial neural network data operation request (B) in advance, the operation of the memory 270 is regarded as a single read access, making it impossible to physically avoid the delay loss. In other words, if the prediction is wrong, the waiting time loss occurs once. After the updated data and address sequence information is stored in the data sequence memory 230, when the processor 260 performs a read access in the same pattern, the next data operation following the first artificial neural network data operation request A can be accurately predicted as the third artificial neural network data operation request (B), so that the loss can be minimized.

[0170] If the prediction of the next address by the internal controller 220 is incorrect, there may be an actual operation error, but there is a case where the processor 260 jumps to another node due to the network structure. For example, there may be a case where the processor 260 deletes / omitted a part of the layer of the artificial neural network or a part of the node. If the artificial neural network is adaptively deformed as described above, the physical waiting time loss cannot be avoided at the first time, but the process is stored in the data sequence memory 230 as updated data and address sequence information, and thereafter, according to the changed configuration, the next data can be prepared in advance without causing any waiting time loss.

[0171] Furthermore, in a state where the data and address sequence information is initialized according to a write request by the artificial neural network operation, even if a read request is made in a different order from the write request, the prediction of the next address by the internal controller 220 may be initially wrong. However, the internal controller 220 can generate the data and address sequence information at the time of the write request and the data and address sequence information at the time of the read request, respectively, and store them in the data sequence memory 230, thereby preparing the next data in advance without waiting time loss.

[0172] If the third address information is inconsistent with the address information of the second artificial neural network data operation request (B′) previously generated / predicted as the next address information in the internal controller 220, the internal controller 220 can compare whether the channel code / identification code of the third artificial neural network data operation request (B) and the first artificial neural network data operation request (A) are the same as the channel code / identification code associated with the previously stored data and address sequence information. At this time, if the previous channel code / identification code based on the second artificial neural network data operation request (B′) and the channel code / identification code of the current third artificial neural network data operation request (B) and the first artificial neural network data operation request (A) are the same as each other, it can be considered that the following situation is: there is an error in the operation of the data and address sequence information itself or the processor 260 dynamically changes the artificial neural network to update the existing data and address sequence information. If the previous channel code / identification code based on the second artificial neural network data operation request (B′) is different from the channel code / identification code of the current third artificial neural network data operation request (B) and the first artificial neural network data operation request (A), it can be interpreted as the following situation: the data and address sequence information is applied differently according to the type of artificial neural network operation, so new data and address sequence information may be generated, and the channel code / identification code of the current third artificial neural network data operation request (B) and the first artificial neural network data operation request (A) can be stored in the data sequence memory 230 as information associated with the new data and address sequence information.

[0173] For example, in the case of a write request, the data is written in a first type of order, but in the case of a read request, the data may be read in a second type of order. During training, the order of write requests and the order of read requests may be the same as each other, but during inference, the order of write requests and the order of read requests may be different from each other, and vice versa (during inference, the order of write requests and the order of read requests are the same as each other, while during training, the order of write requests and the order of read requests are different from each other). Similarly, according to the data domain, in the case of a write request and in the case of a read request, the input data has the same order, but the weight data or activation parameters may have different access data orders depending on the situation, so the data and address order information may be generated in association with the channel code / identification code and may be stored in the data order memory 230. For convenience, the data and address order information adaptively adjusted based on the environment / state information including the channel code / identification code will be represented as adaptive order information.

[0174] When the internal controller 220 uses the data and address sequence information stored in the data sequence memory 230 as an internal memory, it can determine: adaptive sequence information, which is used to generate next address information based on the structure and characteristic information of the artificial neural network associated with the artificial neural network operation; whether the first artificial neural network data operation request is a read request or a write request; and first identification information included in the first artificial neural network data operation request.

[0175] Based on the adaptive order information and the data and address order information stored in the data order memory 230 as an internal memory, the second artificial neural network data operation request can be predicted and the next address information can be generated. For example, when the first / second artificial neural network data operation request is associated with an operation of generating and storing activation parameters, the next address information can be generated in a raster scan order. Alternatively, when the first / second artificial neural network data operation request is associated with an operation of reading the weights of the synapses of the nodes connected to the next layer, it can be read according to the shape of the weight core. In this way, the next address information can be generated based on the adaptive order information according to the type of artificial neural network data operation request and the associated associated artificial neural network operation. Alternatively, when the activation parameters are generated and stored, they are stored in a raster scan order, and when the activation parameters are read from the next layer, they can be read in order based on the shape of the weight core. In the present disclosure, even when the read order and the write order are different, the read order can be predicted by slightly adding a formula for correcting the access address. Therefore, even in this case, it is still possible to maximize the storage bandwidth and minimize or eliminate the waiting time.

[0176] The internal controller 220 may generate information of a current address of the data of the first artificial neural network data operation request stored in the at least one memory as the first address information, using the data and address sequence information stored in the internal memory, based on the first identification information included in the first artificial neural network data operation request and whether the first artificial neural network data operation request is a read request or a write request, even when the first artificial neural network data operation request does not include an address. In the case where the data operation has the same channel code / identification code, the previously stored data and address sequence information may be applied as is. In the training process of the artificial neural network, after the data is initially stored in the memory 270, the weight value may be overwritten in the same address during continuous operation in the training process. In this case, even if the processor 260 does not specify the address value of the memory 270 when overwriting, the data management device 210 may recognize the address value, so that the data operation may be performed. Similarly, after the data is stored in the memory 270 during the training / inference process, the feature map / activation parameter may be overwritten in the same address during continuous operation in the training / inference process. Therefore, when using the data management device 210 according to the present disclosure, the processor 260 can send data operations to the data management device 210 without specifying addresses for data operations in the same kind of data operations. At this time, in order to clearly notify the end of consecutive data operations of the same type, information indicating the end of the last data operation in the same kind of data operations can be included in the data operation request by the processor 260. For the same kind of data operations, if the data operation is sent with address information after being sent without address information, the data operation can be used to notify that the data operation is the last data operation request or the role of the last data operation request is about to come. Alternatively, information about the structure of the artificial neural network can be shared with the data management device 210 in such a way that the data management device 210 itself can determine the end of the same kind of data operations.

[0177] Included in Figure 2 The submodules in the internal controller 220 (access data type detection, next read / write address generator, check legal access) are shown by way of example, and the spirit of the present disclosure is limited to this embodiment. Figure 2 The sub-modules in the internal controller 220 (access data type detection, next read / write address generator, check legal access) are shown for the purpose of illustrating the main functions, and can be implemented in the internal controller 220 using hardware, or can be implemented so that the internal controller 220 can perform the corresponding functions through software.

[0178] Figure 3is a diagram illustrating a data management device 310 for supporting high-speed artificial neural network operations according to another embodiment of the present disclosure.

[0179] refer to Figure 3 , at least one processor 360 sends a data operation request to the data management device 310 via the processor side interface circuit 340, and the data management device 310 sends a memory data operation to at least one memory 370 via the memory side interface circuit 350, which is similar to Figure 2 The data management device 310 includes an internal controller 320 and a data sequential memory 330, which is the same as those of Figure 2 Therefore, redundant description is omitted.

[0180] The data management device 310 of the present disclosure may further include a read data buffer memory, which, when the first artificial neural network data operation request is a read request, performs a pre-read operation on data predicted to be a second artificial neural network data operation request from at least one memory 370 based on next address information, and then stores pre-read data as a result of the pre-read operation.

[0181] At least one of the internal controller 320 or the memory-side interface circuit 340 can send a second memory data operation request to at least one memory 370, can control at least one memory 370 so that at least one memory 370 performs a memory operation corresponding to the second memory data operation request, and can receive data corresponding to the second memory data operation request from at least one memory 370 when the second memory data operation request is a read operation.

[0182] When the next artificial neural network data operation request is not received or delayed after the first artificial neural network data operation request (A), the internal controller 320 may predict the second artificial neural network data operation request (B′) according to the first artificial neural network data operation request (A), and the data management device 310 may send the second artificial neural network data operation request (B′) to the memory 370 and receive the second artificial neural network data operation request (B′) from the memory 370. The read data of the second artificial neural network data operation request (B′) received from the memory 370 is stored in the read data buffer memory 380, and thereafter, the third artificial neural network data operation request (B) as the actual next data operation request is received from the processor 360. When the third artificial neural network data operation request (B) and the second artificial neural network data operation request (B′) are identical to each other, the read data of the second artificial neural network data operation request (B′) stored in the data buffer memory 380 through the pre-read operation may be provided to the processor 360 in response to the third artificial neural network data operation request (B).

[0183] At least one of the internal controller 320 or the memory-side interface circuit 340 may send a second memory data operation request to at least one memory 370, and control at least one memory 370 so that at least one memory 370 maintains a ready state to perform a memory operation corresponding to the second memory data operation request. For example, it is assumed that in a first continuous write operation, an area in the memory 370 is configured in an allocation manner so that continuous write data pieces are allocated and stored in a plurality of areas in the memory 370, and the same kind of overwrite requests for allocated stored data are continuously received. In this case, since the write data must be stored by allocating space in the memory 370 during the continuous write operation, a time difference between the write request of the processor 360 and the actual storage operation of the memory 370 may occur. In this case, the data management device 310 may further include a write data buffer memory 390. The write data buffer memory 390 may store continuous write requests and write data sent from the processor 360, and send the write data to the memory 370, thereby minimizing or reducing to zero the waiting time loss recognized by the processor 360. In addition, the second artificial neural network data operation request (B′) to be followed next based on the first artificial neural network data operation request (A) that has been currently received can be predicted, and the second artificial neural network data operation request (B′) can be sent to the memory 370, so that the memory 370 can be controlled in a manner that makes the memory 370 ready to process the second artificial neural network data operation request B′. At this time, in a state where the memory 370 is not yet ready to process the second artificial neural network data operation request (B′), when a third artificial neural network data operation request (B) that is the same as the second artificial neural network data operation request (B′) is received from the processor 360, the write data buffer memory 390 can temporarily store the third artificial neural network data operation request (B), and when the preparation of the memory 370 to process the third artificial neural network data operation request (B) is completed, the write data buffer memory 390 can send the third artificial neural network data operation request (B) to the memory 370.

[0184] When the write data already stored in the write data buffer memory 390 is stored in the memory 370, the internal controller 320 may erase the write data from the write data buffer memory 390. The internal controller 320 may store the next write operation and the write data received from the processor 360 in the write data buffer memory 390 when free space appears in the write data buffer memory 390, and may directly send the next write operation and the write data received from the processor 360 without passing through the write data buffer memory 390.

[0185] The internal controller 320 may monitor the ratio of free space in the write data buffer memory 390. When the ratio of free space is equal to or greater than a threshold value, the internal controller 320 may notify the processor 360 through an interrupt that the ratio of free space in the write data buffer memory 390 is equal to or greater than the threshold value; or an adjustment may be performed so that the processor 360 can check the ratio of free space in the write data buffer memory 390.

[0186] Figure 4 4 is a diagram showing a situation where a data management method for supporting an artificial neural network operation is executed in a computing system 400 according to yet another embodiment of the present disclosure.

[0187] according to Figure 4 The computing system 400 for supporting artificial neural network operations of the embodiment of the present disclosure shown in is a computing system having at least one processor and at least one memory and supporting artificial neural network operations executed by at least one processor. The computing system 400 includes: a processor-side interface for receiving an artificial neural network data operation request associated with the artificial neural network operation executed by at least one processor; a memory-side interface for sending a memory data operation request to the at least one memory based on the artificial neural network data operation request, and receiving memory data from the at least one memory in response to the memory data operation request; and a data sequence memory for storing data and address sequence information.

[0188] Figure 4 The processors shown in are processors capable of executing artificial neural network operations individually or in conjunction with each other. Figure 4 Not shown, but a processor external to computing system 400 may be used to perform artificial neural network operations.

[0189] Figure 4 The memory shown in the figure may include at least one of an internal memory and an external memory of the computing system 400. The internal memory or the external memory may include various conventional memory devices. For example, SRAM, DRAM, E2PROM, flash memory, or NVRAM (non-volatile RAM) may be included as the internal memory or the external memory of the computing system 400.

[0190] Figure 4 The computing system 400 may include an external memory interface connected to an external memory. The external memory interface may send a memory data operation request to at least one external memory outside the computing system 400, and receive memory data from at least one external memory in response to the memory data operation request.

[0191] exist Figure 2 The configuration and functions shown in the data management device 210 or 310 of FIG. 4 or 3 may be implemented in the form of an independent data management device 411 within the computing system 400, or may be distributed and arranged in various elements within the computing system 400. Figure 2 or Figure 3 The configuration and functions shown in the data management device 210 or 310 are allocated and arranged at any position in the data movement path between the memory unit within the computing system 400 and the artificial neural network processor.

[0192] In one embodiment of the present disclosure, Figure 4 The computing system 400 includes an independent data management device 411 connected to the system bus. In this case, the independent data management device 411 can perform the same Figure 2 In a modified embodiment of the present disclosure, the independent data management device 411 includes Figure 2 Some configurations and functions shown in the data management device 210 or 310 of or 3. Figure 2 The remaining configurations and functions of the data management device 210 or 310 of or 3 may be dispersedly included in a data management device 412 embedded in the system bus architecture, a data management device 413 configured as an artificial neural network processor side interface, a data management device 414 embedded in a wrapper block between the memory interface of the internal memory and the system bus, a data management device embedded in the memory interface of the internal memory (not shown), a data management device 415 embedded in the internal memory, a data management device embedded in a memory interface corresponding to an external memory (not shown), a data management device 416 embedded in a wrapper block between the memory interface of the external memory and the system bus, and / or a data management device 417 embedded in the external memory.

[0193] In another embodiment of the present disclosure, Figure 4 The computing system 400 may be implemented by data management devices 413, 414, 415, 416, and 417 arranged at other locations except the independent data management device 411. Figure 2 or Figure 3 The configuration and functions shown in the data management device 210 or 310. Figure 2 or Figure 3The configuration and functions shown in the data management device 210 or 310 may be dispersedly included in a data management device 412 embedded in the system bus architecture, a data management device 413 configured as an artificial neural network processor side interface, a data management device 414 embedded in a wrapper block between a memory interface of an internal memory and a system bus, a data management device (not shown) embedded in a memory interface of an internal memory, a data management device 415 embedded in an internal memory, a data management device (not shown) embedded in a memory interface corresponding to an external memory, a data management device 416 embedded in a wrapper block between a memory interface of an external memory and a system bus, and / or a data management device 417 embedded in an external memory.

[0194] Figure 4 The computing system 400 may be executed by components dispersedly included in the following devices. Figure 2 or Figure 3 The data management device 210 or 310 of the present invention may be a component in an independent data management device 411, a data management device 412 embedded in the system bus architecture, a data management device 413 configured as an interface on the artificial neural network processor side, a data management device 414 embedded in a wrapper block between the memory interface of the internal memory and the system bus, a data management device (not shown) embedded in the memory interface of the internal memory, a data management device 415 embedded in the internal memory, a data management device (not shown) embedded in a memory interface corresponding to an external memory, a data management device 416 embedded in a wrapper block between the memory interface of the external memory and the system bus, and / or a data management device 417 embedded in the external memory. However, since the computing system 400 includes a memory and a processor therein, it is not limited to the data management device 416 of the present invention. Figure 2 or Figure 3 The data management device 210 or 310 of the computing system 400 is different in that data operations among the processor, the data management devices 411 to 417 and the memory may be performed via a system bus in the computing system 400 .

[0195] The computing system 400 can generate a memory data operation request based on an artificial neural network data operation request, can identify the domain and type of the artificial neural network data operation request based on the identification information included in the artificial neural network data operation request, can generate information on the sequence of data of the artificial neural network data operation request associated with the identification information and information on the addresses of the data of the artificial neural network data operation request stored in the sequence in at least one memory as data and address sequence information, and when the processor-side interface receives a first artificial neural network data operation request, can generate information on the second address of data of the second artificial neural network data operation request following the first artificial neural network data operation request in at least one memory as next address information based on information on the first address of the data of the first artificial neural network data operation request stored in at least one memory and data and address sequence information stored in the data sequence memory.

[0196] At least one of the data management devices 411 to 417 dispersedly arranged in the computing system 400 can send a second memory data operation request to at least one memory, can control at least one memory so that at least one memory performs a memory operation corresponding to the second memory data operation request, and when the second memory data operation request is a read operation, data corresponding to the second memory data operation request can be received from at least one memory. In addition, a data management device (not shown) according to another embodiment of the present disclosure can be dispersedly arranged in an interface, a memory, a bus architecture, and a buffer or a cache memory block in an artificial neural network processor.

[0197] At least one of the data management devices 411 to 417 dispersedly arranged in the computing system 400 can send a second memory data operation request to at least one memory, and can control at least one memory so that the at least one memory maintains a ready state capable of performing a memory operation corresponding to the second memory data operation request.

[0198] At least one of the data management devices 411 to 417 dispersedly arranged in the computing system 400 may include a module that receives a first artificial neural network data operation request and then predicts a second artificial neural network data operation request that is expected to follow the first artificial neural network data operation request based on first identification information contained in the first artificial neural network data operation request.

[0199] At least one of the data management devices 411 to 417 dispersedly arranged in the computing system 400 may include a module which, after receiving a first artificial neural network data operation request, generates a second memory data operation request including next address information based on the first artificial neural network data operation, and sends the second memory data operation request to at least one memory through a memory side interface.

[0200] At least one of the data management devices 411 to 417 dispersedly arranged in the computing system 400 may also include a read data buffer memory (not shown) which performs a pre-read operation on data predicted as a second artificial neural network data operation request from at least one memory based on next address information when the first artificial neural network data operation request is a read request, and then stores pre-read data as a result of the pre-read operation.

[0201] As described above, in the same type of continuous data operation requests (for example, when data of the same domain is overwritten), the address information can be omitted in the data operation request sent from the processor inside or outside the computing system 400. In this case, at least one of the data management devices 411 to 417 dispersedly arranged in the computing system 400 can generate address information about the same type of continuous data operation requests and send it to the memory. In addition, at least one of the blocks related to the buffer memory inside the processor, the system bus, the memory interface, and the inside of the memory can generate address information about the same type of continuous data operation requests. The submodule that generates the address information can be implemented in the form of a hardware block or a block embedded with software.

[0202] Figure 5 is a diagram illustrating an example of data operations between a processor and a memory performed via a data management device 510 in a computing system 500 according to an embodiment of the present disclosure.

[0203] For ease of description, the data management device 510 is shown as a single independent device, but as shown in FIG. Figure 4 As shown, the memory 570 may be dispersedly arranged at various locations in the path through which data is sent and received between the processor 560 and the memory 570 , for example, arranged at the processor 560 , the memory 570 , and the system bus architecture (not shown) in the computing system 500 .

[0204] After receiving the first artificial neural network data operation request 520, the data management device 510 may generate a first memory data operation request 530 including the first address information and send it to the memory 570. In response to the first memory data operation request 530, the memory 570 may send the first memory data 540 to the data management device 510, and the data management device 510 may provide the processor 560 with first artificial neural network data 550 based on the first memory data 540 in response to the first artificial neural network data operation request 520.

[0205] Figure 6 is a diagram illustrating an example of data operations between a processor and a memory performed via a data management device 610 in a computing system 600 according to an embodiment of the present disclosure.

[0206] For ease of description, the data management device 610 is shown as a single independent device, but Figure 4 and 5 As shown, the processor 660 and the memory 670 may be dispersedly arranged at various locations in the path through which data is sent and received between the processor 660 and the memory 670 , for example, arranged at the processor 660 , the memory 670 , and the system bus architecture (not shown) in the computing system 600 .

[0207] Typically, in conventional CPU configurations, prefetching or caching is performed based on temporal locality or spatial locality. Such conventional capture can be: capturing a data block containing data when the data is requested (spatial locality), or evicting data that has been accessed from the cache for a long time (temporal locality).

[0208] The present disclosure illustrates a data management device 610 and a computing system 600 that predicts the next data request based on a data request currently received from a processor 660 based on artificial neural network (ANN) data locality present in an artificial neural network (ANN). .

[0209] In the calculation of artificial neural networks, data locality exists like a conventional CPU, and its characteristics are different from the data locality of a CPU. For a specific example, in the case of training, the artificial neural network reads the input data for training from the memory in sequence and calculates it to change the internal weight value. Once the training is completed using all the learning data, it is considered that 1 epoch of training has been completed, and normal artificial neural network training is repeated for several to hundreds of epochs. Therefore, since during training, the input data is repeatedly read from the memory in the same order several to hundreds of times, data locality exists.

[0210] In the case of weights, after the calculation of the inference process is performed by sequentially reading the weights stored in the memory during training, the weight values ​​are updated by post-processing and written to the memory in sequence. When repeating training, the weight data is read from the memory in the same order, and there is also data locality, where the weight values ​​changed after post-processing are written to the memory in the same order. On the other hand, in the case of inference, the calculation of a given artificial neural network algorithm is performed by reading the weight values ​​from the memory in the same order at each inference. In other words, even in inference, there is data locality because the weight values ​​are read from the memory in the same order for each inference operation.

[0211] A feature map (feature map) or activation map (activation parameter map) is output data from one layer in a multi-layer artificial neural network, which is temporarily stored in a memory and used as an input feature map in the operation of the next layer. The order in which the input feature map is read from the memory during the operation of the next layer is the order in which the feature map or activation map data output from the previous layer is first stored in the memory.

[0212] The order of accessing data may be different from the above order depending on the structure of a specific artificial neural network, the operation method of a processor processing the artificial neural network, etc. However, it can be schematically reasoned based on the above data order information, and the changed access pattern is constant under the corresponding artificial neural network structure and processor operation method. Therefore, even in this case, there is clear data locality.

[0213] The present disclosure proposes that there is clear data locality in the operation of an artificial neural network, and by using this, a data management device is proposed, which can minimize or eliminate the delay time caused by memory access when a processor with superior performance compared to the memory asks the memory for data required for the operation of the artificial neural network. In the proposed data management device, the processor predicts the data required for future operations based on the above-mentioned artificial neural network data locality, and pre-fetches it to a memory device (e.g., a cache) adjacent to the processor, so as to provide the minimum delay time for the data when the processor requests the corresponding data. In addition, in the case of overwriting updated weight values ​​in the memory during training and performing a write operation of feature map data calculated and output from a layer on the memory during inference, the proposed memory management device can automatically generate a write address based on the above-mentioned artificial neural network data locality, thereby simplifying and shortening the actions of the processor.

[0214] After receiving the first artificial neural network data operation request 621, the data management device 610 may generate a first memory data operation request 631 including the first address information and send the first memory data operation request 631 to the memory 670. In response to the first memory data operation request 631, the memory 670 may send the first memory data 641 to the data management device 610, and in response to the first artificial neural network data operation request 621, the data management device 610 may provide the first artificial neural network data 651 based on the first memory data 641 to the processor 660.

[0215] After receiving the first artificial neural network data operation request 621 , the data management device 610 may generate a second memory data operation request 632 including next address information based on the first artificial neural network data operation 621 , and send the second memory data operation request 632 to the memory 670 .

[0216] When the second artificial neural network data operation request 622 following the first artificial neural network data operation request 621 is not sent from the processor 660 or is delayed, the data management device 610 can predict the second memory data operation request 632 including the next address information, taking into account the characteristics of the artificial neural network, the characteristics of the artificial neural network operation, and the characteristics of the artificial neural network data operation.

[0217] The predicted second memory data operation request 632 is sent to the memory 670 together with the predicted next address information, and in response to the predicted second memory data operation request 632, the memory 670 can send the second memory data 642 to the data management device 610, and the data management device 610 can temporarily store the second memory data 642 in a read data buffer memory therein.

[0218] Thereafter, when the third artificial neural network data operation request 622 that actually follows the first artificial neural network data operation request 621 is transmitted as a predetermined time has passed from the processor 660 or the obstacle has been resolved, the data management device 610 verifies whether the actually received third artificial neural network data operation request 622 and the predicted second memory data operation request 632 correspond to each other. When it is confirmed that the actually received third artificial neural network data operation request 622 and the predicted second memory data operation request 632 correspond to each other without error, the data management device 610 may provide the second memory data 642 as the third artificial neural network data 652 to the processor 660 in response to the third artificial neural network data operation 622.

[0219] In any single memory read, the latency of ordinary memory exceeds several clock cycles. This causes the processor to enter an idle state until the necessary data is provided, thereby preventing the processor from performing actual operations. This is a limitation that also applies to the memory 660 and processor 670 of the present disclosure.

[0220] After each data group used in the artificial neural network is written to the memory 670, the order in which the processor 660 reads the data of the corresponding data groups for training and inference generally follows the order in which the data blocks of the corresponding data groups are first written to the memory 670, determined based on the order in which they are first written to the memory 670, or determined based on the order in which they were previously written to the memory 670.

[0221] When the system 600 or processor 660 accesses any data included in a particular data group of the artificial neural network (e.g., input data, weights, and activation parameters used for training), the data management device 610 can predict which data in the data group the processor 660 or system 600 will request access to.

[0222] When the processor 660 resumes the artificial neural network data operation after the continuous artificial neural network data operation stops for a certain period of time, the first data requested by the resumed data operation is processed as any single memory read operation according to the prior art, and thus the physical waiting time cannot be avoided. At this time, if the read data buffer memory (a high-speed storage device that can directly provide data without delay) included in the data management device 610 of the present disclosure has previously stored an amount of data that can continuously provide a time period equal to the waiting time period caused by any single memory read operation through a pre-read operation, the processor 660 can immediately perform the operation without delay by providing the processor 660 with data delay without the waiting time for any memory read operation, so that the overall operation performance can be improved.

[0223] It can be determined by considering the information related to the artificial neural network operation (channel code / identification code) included in the first artificial neural network data operation request 621 and the sequential address mapping process on the memory 670 that has been previously performed during the continuous write operation: the data management device 610 predicts the process of the second memory data operation request 632 that will occur immediately after the first artificial neural network data operation request 621 that has been received, and the process can be performed using information about the storage data access pattern of the artificial neural network operation.

[0224] When the processor 660 starts the training or reasoning operation of the artificial neural network, it starts to read the data of each data group, and may frequently repeat the interruption / recovery of data reading during the entire operation. During the entire operation, the node value of each layer of the artificial neural network is output and the operation is completed. As described above, when the reading is resumed after the reading is interrupted, since the first access to be resumed becomes an arbitrary single access, there is inevitably a significant (great) waiting time in the prior art.

[0225] When the read operation of the next necessary data is temporarily stopped for any reason while the processor 660 continues to read data, the data management device 610 and the computing system 600 of the present disclosure read the data predicted as the next necessary data from the memory 670 in advance and store the data (date) in a read data buffer memory (not shown) therein. At this time, during the waiting period caused when the read operation of the processor 660 is resumed after the stop, the data previously stored in the read data buffer memory can replace the memory 670 to provide data to the processor 660. At the same time, the computing system 600 or the data management device 610 accesses the memory 670 and reads in advance the data predicted to follow the data in the read data buffer memory, thereby filling the space from which the data provided to the processor 660 has been eliminated. In view of the presence of the processor 660, it should be recognized that, except for the physical delay time (propagation delay) of the data, the memory 670 operates with a waiting time of 0 during the read operation.

[0226] The depth of the read data buffer may be the number of clock cycles required to output the actual data from the memory 670 from the time the address is provided to the memory 670. According to one embodiment, the optimal depth may vary depending on the artificial neural network model, the type of operating processor 660 and memory 670, and the like. If the buffer depth is large, it is beneficial to improve the operating performance, but based on a single memory cell, a buffer memory with a delay of 0 or high-speed operation requires a greater hardware cost and higher power consumption compared to a memory of large capacity DDR / SRAM / FLASH. Therefore, it may be necessary to weigh increasing the depth of the pre-fetch buffer. The depth of the read data buffer memory that can achieve maximum performance may vary depending on the operating performance of the processor 660, data timing, the waiting time of the memory 670, performance characteristics, and the characteristics of the artificial neural network model as the operating target.

[0227] When the computing system 600 or the processor 660 resumes reading the next data of the previously read data after an interruption in reading the continuous data, the data management device 610 or the computing system 600 provides the data stored in the buffer, thereby causing the computing system 600 or the processor 660 to recognize the memory 670 as operating with a latency of 0. At the same time, the data management device 610 or the computing system 600 can access the memory 670 to read the next data of the last data stored in the buffer, receive the data after the inherent latency of the memory 670, and store it in the buffer or send it to the processor 660.

[0228] Meanwhile, the reasons for stopping the artificial neural network data operation between the processor 660 and the memory 670 may include the following situations:

[0229] 1) When no operation is performed between the processor 660 and the memory 670,

[0230] 2) A case where an operation is performed between the processor 660 and the memory 670 on data other than data associated with the artificial neural network,

[0231] 3) Although no operation is performed between the processor 660 and the memory 670, the processor 660 performs data operations with another memory (for example, when weight data is stored in the corresponding memory 660 and activation parameters or feature map data is stored in another memory, the processor 660 may perform data access alternately between the corresponding memory 670 and the other memory. As another example, when an operation is performed on data other than data associated with another memory and an artificial neural network, the processor 660 may not access the memory 670),

[0232] 4) The situation where the processor 660 has all the data required to perform the artificial neural network processing operation and temporarily does not need the data to perform the required operation (this situation may be considered as a reason to possibly stop the continuous memory access).

[0233] At the same time, the computing system 600 or the data management device 610 of the present disclosure can determine whether the first artificial neural network operation associated with the first artificial neural network data operation request 621 has been completed based on the order in which data is stored in a data sequence memory (not shown) included therein and information about the addresses at which data is stored in at least one memory in that order, and if the first artificial neural network operation has not been executed, the operation of the present disclosure can be performed, and if the first artificial neural network operation has been completed, the operation of the present disclosure can be omitted.

[0234] The data management method for supporting artificial neural network operations according to an embodiment of the present disclosure is a method performed in a computing system 600 having at least one processor 660 and at least one memory 670 and supporting artificial neural network operations performed by the at least one processor. The data management method may include: receiving an artificial neural network data operation request associated with an artificial neural network operation performed by the at least one processor 660; sending a memory data operation request based on the artificial neural network data operation request to at least one memory 670, and receiving memory data from at least one memory 670 in response to the memory data operation request; and generating a memory data operation request based on the artificial neural network data operation request.

[0235] Figure 7 is an operational flow chart showing some processes of a data management method according to an embodiment of the present disclosure.

[0236] As part of the data management method of the present disclosure, generating a memory data operation request based on an artificial neural network data operation request may include: in step S710, identifying the domain and type of the artificial neural network data operation request based on the identification information contained in the artificial neural network data operation request; in step S720, generating the sequence of data of the artificial neural network data operation request associated with the identification information as data and address sequence information and information on the addresses of the data of the artificial neural network data operation request stored in this sequence in at least one memory; and in step S730, storing the data and address sequence information in a data sequence memory inside the computing system.

[0237] Figure 8 is an operational flow chart showing some processes of a data management method according to an embodiment of the present disclosure.

[0238] As part of the data management method of the present disclosure, generating a memory data operation request based on an artificial neural network data operation request may include: when a first artificial neural network data operation request is received in step S810, in step S820, generating information of a second address of data of a second artificial neural network data operation request following the first artificial neural network data operation request in at least one memory as next address information based on information of a first address of data of the first artificial neural network data operation request stored in at least one memory and data and address sequence information stored in a data sequence memory.

[0239] Fig. 9 is an operational flow chart showing some processes of a data management method according to an embodiment of the present disclosure.

[0240] As part of the data management method of the present disclosure, generating a memory data operation request based on an artificial neural network data operation request may include: after receiving a first artificial neural network data operation request in step S910, predicting the second artificial neural network data operation request that is expected to follow the first artificial neural network data operation request based on first identification information included in the first artificial neural network data operation request in step S920; and predicting information of a second address for storing data of the second artificial neural network data operation request in at least one memory based on first address information and data and address sequence information in step S930.

[0241] Fig.10 is a diagram showing data and address sequence information according to an embodiment of the present disclosure.

[0242] Fig.10 An implementation of data and address sequence information when a fully dedicated memory is used for artificial neural network operations is shown.

[0243] Reference Fig.10 , since a given memory is used only for artificial neural network operations, the address space of the given memory is clearly allocated / divided into memory areas for each of input data, weights, and feature maps. Therefore, the data sequential memory can store the start address and end address of the memory area for each data group, and store the memory address for the next read based on the addresses of the consecutive reads that have been performed so far, so that it can predict and respond to the operation after the data operation of the artificial neural network required by the system / processor.

[0244] Fig.10 The system address map of conceptually indicates that input data, weights, and feature maps are allocated in partitioned areas of memory.

[0245] At this time, 0x1 000000 (which is the end address of the input data stored in the data sequential memory) can be represented as a value obtained by adding 1 to the last address of the actual input data. Similarly, the end address of the weight data can also be stored in the data sequential memory as 0x2000000, which is a value obtained by adding 1 to the last address of the actual weight data.

[0246] Fig.11 is a diagram showing data and address sequence information according to an embodiment of the present disclosure.

[0247] Fig.11 An embodiment is shown in which a large linear memory region for artificial neural network operations is allocated within the address space for each use of the memory.

[0248] In addition to artificial neural networks, the system can also use one memory for multiple system tasks. In this case, Fig.11 It is shown that each data domain used for the artificial neural network operation is clearly allocated by use.

[0249] The system address map shows the input data region, weight region, and feature map region, which are clearly separated within the entire address space of the memory. Accordingly, the data sequential memory can fully function by storing the start address and end address of each data domain and storing the next memory address to be read based on the addresses of the consecutive reads that have been performed so far.

[0250] Fig.12 is a diagram showing data and address sequence information according to yet another embodiment of the present disclosure.

[0251] Fig.12 An embodiment is shown in which a fragmented memory area is allocated for each data domain.

[0252] A system with an operating system (OS) requires various different types of tasks to use memory, and when each of these tasks uses memory, the unallocated memory area is fragmented. The memory area is dynamically allocated, and after its use, memory fragments / fragments are formed during the release process. In this case, it is difficult to allocate a large-capacity single memory area for artificial neural network operations. In this case, fragmented memory areas must be allocated to perform artificial neural network processing. The data sequence memory stores the connection relationship between these fragmented memory areas. That is, the data and address sequence information includes information on how to connect the fragmented memory areas for one data domain.

[0253] The system address diagram shows a situation where a fragmented memory area within a memory address space is used for a data domain. The area shown in white in the system address diagram is the memory area allocated for input data. Fig.12 The system address diagram exemplarily describes input data, but the method of implementing the present disclosure is similar in the case of weight data or feature map data.

[0254] The data and address sequence information starts with storing the start and end addresses of each fragmented memory area. At this point, the address of the next read operation can be predicted based on the addresses of the read operations that have been executed continuously so far. When the address of the current read operation reaches the last address of any fragmented memory area, access to the next fragmented memory area will be prepared in advance for the next operation.

[0255] A method of storing a start address and an end address in a data sequential memory according to other embodiments of the present disclosure will be described below.

[0256] In the first method, when the processor initially calculates the artificial neural network information by itself and the processor knows information about the allocated memory area, the processor can access the data sequential memory like a special function register (SFR) and directly register the start address and the end address.

[0257] In the second method, when the processor first stores data in each channel (data domain, which means a kind of artificial neural network data), the data sequence memory can record the time when the processor first stores the data of the corresponding data group as the first starting address. The processor continuously performs monitoring when storing the data of the data group, and if the write address corresponds to a simple word increase (for example, when the address is increased by 4 in a 32-bit system), no special measures are taken, but if the write address does not correspond to the simple word increase, the address value where the simple increase has been terminated is recorded as the end address of a memory area, and the jump address value that is not a simple increase is recorded in the start address of the next entry. After that, it continues to monitor whether it corresponds to the simple word increase, and when the write operation of the last data is performed, the corresponding address is written to the end address of the last recorded start address entry, and the data and address sequence information stored in the data sequence memory are completed.

[0258] Fig.13 is a diagram illustrating a method of expressing channel information / identification information according to various embodiments of the present disclosure.

[0259] In artificial neural network operations, data fields / data groups must be accurately represented for write access and read access. Figure 1 3, the following three implementation schemes are used to express a process in which a processor and a data management device share information about a data group.

[0260] (1) Data group flag: The access data group can be expressed for each access by using a specific interface I / O signal.

[0261] (2) Address: In the address area there will be an implementation scheme for classifying channels.

[0262] (3) Register access: There may be an implementation in which the processor records information about the channel in a special function register (SFR).

[0263] (1) For example, in the implementation of the data group flag, a 2-bit signal can be added to the I / F using a processor. At this time, if the special signal is 00, it can be a code indicating access to data other than ANN data. If it is 01, it can be a code indicating that the corresponding data group belongs to input data. If it is 10, it can be a code indicating weight data. If it is 11, it can be a code indicating feature map data.

[0264] (2) In the implementation of this address, when the address is sent from the processor side, an implementation in which the memory area to be accessed is accessed differently is possible. Fig.14 This embodiment is described in detail.

[0265] Fig.14 is a diagram showing a case in which channel information / identification information and data and address sequence information are associated according to an embodiment of the present disclosure.

[0266] refer to Fig.14 , shows the process of allocating a memory area for each data group in the system address map. MEMORY_SIZE is the size of the memory area for each data group.

[0267] The processor allocates memory regions of the memory to the system address map in various multiples, each region representing a specific data group. For example, channel 1 can be non-ANN data. Channel 2 can be input learning data, channel 3 can be weight data, and channel 4 can be feature map data.

[0268] exist Fig.14 In the embodiment shown in , the data management device can determine which channel the access is to be made by examining the most significant bit (MSB) of the address.

[0269] According to an embodiment of the present disclosure, it can be implemented so that only one data group can be stored in the same nth address n except for the MSB as the address in the memory area. This situation is a method to avoid confusion between different data groups.

[0270] For example, when some data is stored in base address (offset) + memory_size + address n, it is a method of performing adjustment so that other data is not stored in base address (offset) + channel × memory_size + address n.

[0271] Reference again Fig.13 , the implementation scheme of (3) register access will be described.

[0272] A register capable of representing a group of currently accessed data is formed in the data management device and is assigned to the system memory map as an address value. Before accessing data of a data group, the processor first writes the data group representation value into the register. In this way, data group or channel information can be shared between the processor and the data management device.

[0273] exist Figure 1 In the embodiment of 3, it has been described that there are four channels, but this is only one embodiment, and the spirit of the present invention is not limited thereto. In the case of 4 channels, if the channel code is 00, the channel can be represented as Channel 1; if the channel code is 01, the channel can be represented as Channel 2; if the channel code is 10, the channel can be represented as Channel 3; if the channel code is 11, the channel can be represented as Channel 4. However, this is only one embodiment, and the spirit of the present invention is not limited thereto.

[0274] As described above, the biggest influence on the operation of the artificial neural network is whether the data belongs to any data group among the input data, weight data, or feature map data. Meanwhile, for the weight data or feature map data, since the order of data operations can be changed according to whether it is a read operation or a write operation, or whether it is in the training process or in the inference process, the channel code can be subdivided into more than 2 bits to indicate the order of adaptive adjustment.

[0275] exist Figures 2 to 14 The embodiments of the present disclosure shown in the present disclosure propose a data management method and a method for supporting artificial neural network operations, which can adaptively respond by monitoring artificial neural network data operations associated with the artificial neural network operations by the computing system or data management device of the present disclosure even when the continuous mode of the artificial neural network data operations is changed.

[0276] At the same time, Figures 2 to 14 The embodiment of the present disclosure shown in is combined with an embodiment (in which the process that will perform artificial neural network operations is analyzed in advance, the data locality and sequential patterns of artificial neural network data operations are extracted, and the memory configuration is reconfigured in software) so that artificial neural network operations can be supported in more ways.

[0277] Fig.15 is a diagram illustrating a data management device 1500 adaptively reconfigured to support artificial neural network operations according to an embodiment of the present disclosure.

[0278] refer to Fig.15, the data management device 1500 includes at least one processor 1510 and a reconfigurable memory 1550. In this case, the data management device 1500 may correspond to a computing system. The data management device 1500 may be coupled to a communication interface or a user interface module not shown in the figure, may receive user input and provide a response to the user, and may send and receive program instruction codes and / or data to and from an external computing system, processor, or memory.

[0279] At least one processor 1510 can analyze a process in which an artificial neural network operation is to be performed by a program instruction executed in the at least one processor 1510, and extract a sequential pattern of an artificial neural network data operation associated with the artificial neural network operation. At this time, information such as the type, characteristics, and purpose of the artificial neural network is given, and the at least one processor 1510 can extract the sequential pattern of the artificial neural network data operation in advance by simulating the artificial neural network operation. In the at least one processor 1510, a configuration for extracting the sequential pattern is shown as an artificial neural network analysis module 1520.

[0280] In this case, the artificial neural network analysis module 1520 in the at least one processor 1510 may extract the sequential pattern based on the artificial neural network data locality and the artificial neural network operation of the artificial neural network data operation. Figure 6 The meaning of data locality in artificial neural networks has been described in the implementation scheme.

[0281] At this time, the artificial neural network analysis module 1520 in at least one processor 1510 can extract the sequential pattern based on the following situations: whether the artificial neural network data operation request is a request for input data for training, a request for a weight assigned to each synapse of the artificial neural network, or a request for an activation parameter, which is a result of an operation from an output node of one layer to an output node of the next layer, and whether the currently received data request corresponds to the artificial neural network data operation request.

[0282] At least one processor 1510 can generate artificial neural network memory configuration information based on the sequential pattern of artificial neural network data operations through program instructions executed in at least one processor 1510. The artificial neural network memory configuration information may include: information required for artificial neural network data operations, such as the size, dimension and arrangement of the memory. In at least one processor 1510, the configuration for generating the artificial neural network memory configuration information is shown as a memory configuration generation module 1530.

[0283] The artificial neural network memory configuration information may include data between the processor side that executes the artificial neural network operation and the memory that stores the data, the mapping between the addresses of the stored data, and the mapping between the addresses of the stored data as described above. Figures 2 to 14 The implementation scheme includes sequence information similar to data and address sequence information.

[0284] The reconfigurable memory 1550 may be controlled so that the at least one processor 1510 may apply the artificial neural network memory configuration information to the reconfigurable memory 1550 through program instructions executed in the at least one processor 1510. The configuration for applying the artificial neural network memory configuration information to the reconfigurable memory 1550 is shown as a memory configuration application module 1540 in the at least one processor 1510.

[0285] The artificial neural network analysis module 1520, the memory configuration generation module 1530, and the memory configuration application module 1540 may be implemented as separate hardware, may be implemented as software, and may be implemented by a combination of hardware and software.

[0286] As in Figure 2 In the embodiment described in the embodiment, the order of making write requests and the order of making read requests can be consistent with each other, and when making a write request, the data operation can be performed in the order of the first type, and when making a read request, the data operation can be performed in the order of the first type. When performing data operations, the data operations can be performed in the order of the second type.

[0287] The artificial neural network analysis module 1520 or the memory configuration generation module 1530 in at least one processor 1510 can extract the runtime sequential pattern of the artificial neural network data operation based on the structure and characteristic information of the artificial neural network related to the artificial neural network operation, the sequential pattern, and whether the artificial neural network data operation is a read request or a write request.

[0288] The reconfigurable memory 1550 can store data required for artificial neural network operations, and can provide artificial neural network data operation requests corresponding to artificial neural network operations as responses to artificial neural network operations. At this time, the operation of the reconfigurable memory 1550 can be controlled by at least one processor 1510, and at least one processor 1510 can pre-execute or prepare the next artificial neural network data operation while the current artificial neural network data operation is being executed. The process in which at least one processor 1510 can pre-execute or prepare the next artificial neural network data operation while the current artificial neural network data operation is being executed can refer to the above-mentioned Figures 2 to 14 implementation plan to achieve this.

[0289] Fig.16 is a diagram illustrating a data management device adaptively reconfigured to support artificial neural network operations according to an embodiment of the present disclosure.

[0290] refer to Fig.16 , the data management system 1600 may include a data management device 1610 . The data management device 1610 includes a controller 1612 and a reconfigurable memory 1650 .

[0291] The artificial neural network analysis module 1620 outside the data management device 1610 can extract the sequential pattern of artificial neural network data operations based on the process of performing artificial neural network operations. Figure 1 The communication interface not shown in 6 receives the sequential pattern of artificial neural network data operations from the artificial neural network analysis module 1620.

[0292] The memory configuration generation module 1630 of the controller 1612 can generate the artificial neural network memory configuration information based on the sequential mode through program instructions executed in the controller 1612. The artificial neural network memory configuration information may include data and address sequence information or information equivalent thereto. The controller 1612 can generate the artificial neural network memory configuration information based on the physical configuration information of the reconfigurable memory 1650 and the sequential mode.

[0293] The memory configuration generation module 1630 of the controller 1612 can extract the runtime sequence pattern of the artificial neural network data operation based on the structure and characteristic information of the artificial neural network associated with the artificial neural network operation, the sequence pattern, and whether the artificial neural network data operation is a read request or a write request.

[0294] That is, the controller 1612 is based on the sequential pattern extracted by the artificial neural network analysis module 1520, but the controller 1612 can access the reconfigurable memory 1650 based on the run-time sequential pattern adaptively adjusted when actually providing the artificial neural network data operation corresponding to the artificial neural network operation.

[0295] When receiving an artificial neural network operation, the controller 1 612 can cooperate with the reconfigurable memory 1 650 based on the run-time sequential mode and provide the artificial neural network data operation as a response to the artificial neural network operation. Figures 2 to 14 implementation scheme to achieve this process.

[0296] The memory configuration application module 1640 of the controller 1612 may control the reconfigurable memory 1650 so as to apply the artificial neural network memory configuration information to the reconfigurable memory 1650 through program instructions executed in the controller 1612 .

[0297] exist Figure 1 5 and Figure 1 In the embodiment of FIG. 6, the reconfigurable memories 1550 and 1650 may be formed as physically universal memories, or may be formed as a combination of universal memories. At this time, the physical configuration of the reconfigurable memories 1550 and 1650 is not changed, but the reconfigurable memories 1550 and 1650 may be reconfigured in a logical manner or based on software. That is, the size of the logical sectors, blocks, planes, etc. inside the reconfigurable memories 1550 and 1650 may be reconstructed based on the memory configuration information. The processor 1510 or the controller 1612 may pre-store typical memory configuration information corresponding to several types of typical artificial neural network operations, may select one of the plurality of memory configuration information previously stored based on a user command, and may apply the selected one to the reconfigurable memories 1550 and 1650.

[0298] The artificial neural network operation may be performed by a separate computing system or processor, or may be performed by neural mimicking hardware such as the recently proposed neuromorphic. The data management devices 1500 and 1610 including reconfigurable memories 1550 and 1650 according to the embodiments of the present disclosure may be arranged on site by combining with hardware or software that performs artificial neural network operations. At this time, the processor 1510 or the controller 1612 may provide the user with options for memory configuration information via a user interface (not shown), and if one of the options for memory configuration information is selected by a user command, the selected option may be applied to the reconfigurable memories 1550 and 1650. When the processor 1510 or the controller 1612 provides the user with options for memory configuration information through a user interface (not shown), information such as the type of artificial neural network corresponding to each option and the type of problem to be solved may be provided to the user together. The process of determining the memory configuration information by a user command may be performed in various situations, such as when the data management devices 1500 and 1610 are initialized in a factory, deployed on site, or readjusted on site.

[0299] At this time, the options of the memory configuration information can include a portion of the run time sequence mode. For example, an option can be provided so that the user can select the memory configuration information according to whether to use the artificial neural network to perform learning / training or to use the already learned data and the artificial neural network to infer the solution of the new problem.

[0300] Additionally, a menu is provided to enable the user to select or input the size or dimension of the artificial neural network, and the logical dimensions of the reconfigurable memories 1550 and 1650 may be determined based on user commands input through the user menu.

[0301] The memory configuration information of the reconfigurable memories 1550 and 1650 in the runtime of executing the artificial neural network operation may be applied statically or dynamically. The memory configuration information determined once may be applied statically, and if an error occurs or data movement is required in a specific area of ​​the reconfigurable memories 1550 and 1650 at runtime, the processor 1510 or the controller 1612 may dynamically change the physical address of the reconfigurable memories 1550 and 1650, and the memory configuration information may be dynamically changed and applied at runtime.

[0302] The data management devices 1500 and 1610 may further include a read-only memory (ROM) associated with the memory configuration generation modules 1530 and 1630 and store a template of memory configuration information. Although described as ROM here, known programmable memories such as programmable ROM, EPROM, EEPROM, flash memory, etc. may be used.

[0303] refer to Fig.15 and Fig.16 , the artificial neural network analysis modules 1520 and 1620 extract sequential patterns by simulating or analyzing the execution of artificial neural network operations in advance, and in this case, sequential patterns including branches, jumps or flows of data can be extracted. Fig.15 and Fig.16 The implementation scheme is provided as a data cache for artificial neural network data operations, and in addition to the case of simply performing data prefetching, detailed optimization can be performed according to software configuration.

[0304] exist Fig.15 and Fig.16 In the embodiment of the present disclosure, a reconfigurable memory platform is proposed, which can adjust the memory configuration of the data management device 1500 and 1610 for supporting the operation of the artificial neural network based on the artificial neural network used by the user and the problem to be solved. In addition, the reconfigurable memory platform can be adaptively optimized through software analysis of the artificial neural network. The data management device 1500 and 1610 for supporting the operation of the artificial neural network according to the embodiment of the present disclosure can be understood as a logically reconfigured software or memory compiler.

[0305] The data management method for supporting the operation of artificial neural networks according to the embodiment of the present disclosure can be implemented in the form of program instructions that can be executed by various computer devices and recorded in a computer-readable medium. The computer-readable medium can include program instructions, data files, data structures, etc., alone or in combination. The program instructions recorded on the medium can be specially designed and configured for the present disclosure, or can be known and usable by technicians in the field of computer software. Examples of computer-readable recording media include magnetic media (such as hard disks, floppy disks, and tapes), optical media (such as CD-ROMs and DVDs), magneto-optical media (such as floppy disks), and hardware devices (such as ROM, RAM, flash memory, etc.) that are specially configured to store and execute program instructions. Examples of program instructions include not only machine language codes, such as those generated by compilers, but also high-level language codes that can be executed by computers using interpreters, etc. The above-mentioned hardware devices can be configured to operate as one or more software modules to perform the operations of the present disclosure, and vice versa.

[0306] However, the present disclosure is not limited or constrained by the embodiments. In the various drawings, the same reference numerals represent the same components. The length, height, size, width, etc. introduced in the embodiments and drawings of the present disclosure may be exaggerated to help understanding.

[0307] In the above, although the present disclosure is described by specific contents, embodiments and drawings such as specific components, they are only provided to help the overall understanding of the present disclosure. Therefore, the present disclosure is not limited to the embodiments. According to this specification, various modifications and changes can be made by those skilled in the art to which the present disclosure belongs.

[0308] Therefore, the spirit of the present disclosure should not be limited to the above-mentioned embodiments, and the appended claims and all modifications that are equivalent or equivalent to the claims fall within the scope and spirit of the present invention. Industrial Applicability

[0309] A data cache or data management device is disclosed, which is used to cache data between at least one processor and at least one memory, and supports artificial neural network (ANN) operations performed by at least one processor. The data cache device or data management device may include an internal controller, which is used to predict the next data operation request based on the ANN data locality of the ANN operation. The internal controller monitors the data operation request associated with the ANN operation from the data operation request actually made between the at least one processor and the at least one memory, thereby extracting the ANN data locality of the ANN operation.

Claims

1. A data management device, comprising: a data sequence memory configured to store data-address sequence information; as well as at least one buffer memory configured to cache data of a data operation of the artificial neural network; and The data management device is configured as follows: receiving a first memory read request from a processor; predicting a second memory read request to be requested after the first memory read request by referring to the data-address sequence information; caching data from at least one memory corresponding to the second memory read request in the at least one buffer memory; The data corresponding to the second memory read request cached in the at least one buffer memory is provided to the processor.

2. The data management device according to claim 1, further comprising A read / write address generator module is configured to generate the data-address sequence information.

3. The data management device according to claim 1, The first memory read request is an artificial neural network data operation request.

4. The data management device according to claim 1, The data-address sequence information is generated based on the artificial neural network data position.

5. The data management device according to claim 1, The processor is one of a CPU, a GPU, a network processor (NNP), and a processor customized for data operations of an artificial neural network.

6. The data management device according to claim 1, wherein the data management device is further configured to determine whether the second memory read request and a third memory read request generated by the processor are the same; and wherein the data corresponding to the second memory read request is provided to the processor in response to the third memory read request.

7. The data management device according to claim 1, wherein the data management device is further configured to determine whether the second memory read request and a third memory read request generated by the processor are the same; and The data corresponding to the second memory read request is transferred to the at least one memory before receiving the third memory read request.

8. The data management device according to claim 1, further comprising A check legal access submodule is configured to check whether the address information of the second memory read request and the third memory read request match and The data management device is further configured to determine whether the second memory read request and a third memory read request generated by the processor are the same.

9. The data management device according to claim 1, further comprising a first interface circuit in communication with the processor; and A second interface circuit is in communication with the at least one memory.

10. The data management device according to claim 1, in, An input data storage area, a weight storage area, and a feature map storage area are allocated to the at least one memory.

11. A data management device, comprising: a data sequence memory configured to store data-address sequence information; as well as The data management device is configured as follows: receiving a first memory read request from a processor; predicting a second memory read request to be requested after the first memory read request by referring to the data-address sequence information; and At least one memory storing data corresponding to the second memory read request is controlled to maintain a ready state in which the second memory read request can be executed.

12. The data management device according to claim 11, further comprising A read / write address generator module is configured to generate the data-address sequence information.

13. The data management device according to claim 11, The first memory read request is an artificial neural network data operation request.

14. The data management device according to claim 11, The data-address sequence information is generated based on the artificial neural network data position.

15. The data management device according to claim 11, The processor is one of a CPU, a GPU, a network processor (NNP), and a processor customized for data operations of an artificial neural network.

16. The data management device according to claim 11, The data management device is further configured to: determining whether the second memory read request and a third memory read request generated by the processor are the same; and The data corresponding to the second memory read request is provided to the processor.

17. The data management device according to claim 11, in, The data stored in the at least one memory includes at least one of weights, activation maps, and feature maps.

18. The data management device according to claim 11, The data includes data of a multi-layer artificial neural network, and temporarily storing in the at least one memory an output feature map from a first layer of the multi-layer artificial neural network, The output feature map is used as the input feature map of the second layer of the multi-layer artificial neural network.

19. The data management device according to claim 11, further comprising At least one buffer memory configured to cache data of data operations of the artificial neural network.

20. An apparatus comprising: A data sequence memory storing data-address sequence information; and at least one buffer memory configured to cache data of a data operation of the artificial neural network; and The device is configured as follows: receiving a first memory read request via a first interface circuit; predicting a second memory read request to be requested after the first memory read request by referring to the data-address sequence information; caching data from the memory corresponding to the second memory read request into the buffer memory; The cache data corresponding to the second memory read request cached in the buffer memory is transmitted to the processor.

21. The device according to claim 20, The device is further configured to: A determination is made as to whether the second memory read request is consistent with a third memory read request generated by the processor.

Images