Storage device that stores ai model and operating method thereof

Abstract

The storage device may include: a memory for storing a plurality of data units, each containing one or more parameters of an artificial intelligence model; and a controller that receives structural information for an artificial intelligence model from outside the storage device, determines a plurality of data units based on the structural information, and determines whether to cache each of the plurality of data units in a cache.

Description

Technology Field

[0001] The embodiments of the present disclosure relate to a storage device for storing an artificial intelligence model and a method of operating the same. Background Technology

[0002] A storage device is a device that stores data based on requests from external devices, such as computers, mobile terminals like smartphones and tablets, or various electronic devices.

[0003] The storage device may further include a controller for controlling memory (e.g., volatile memory / non-volatile memory), and such a controller may receive a command from an external device and, based on the received command, execute or control operations to read, write, or erase data in the memory included in the storage device.

[0004] Storage devices can store artificial intelligence models. To enhance the performance of artificial intelligence, it is necessary to reduce the time required to read artificial intelligence models stored in the storage device. In particular, when multiple processors perform parallel read operations on artificial intelligence models stored in the storage device, it is necessary to improve read performance by reducing the time required for the read operation. The problem to be solved

[0005] Embodiments of the present disclosure may provide a storage device and a method of operation thereof that can improve read performance for an artificial intelligence model stored in a storage device.

[0006] The problems of the embodiments of the present disclosure are not limited to those mentioned in this specification, and other problems not mentioned will be clearly understood by those skilled in the art from the description below. means of solving the problem

[0007] Embodiments of the present disclosure may provide a storage device comprising: a memory for storing a plurality of data units each comprising one or more parameters of an artificial intelligence model; and a controller for receiving structural information for an artificial intelligence model from outside the storage device, determining a plurality of data units based on the structural information, and determining whether to cache each of the plurality of data units in a cache.

[0008] Embodiments of the present disclosure may provide a method of operation of a storage device comprising: receiving structural information for an artificial intelligence model from outside the storage device; determining a plurality of data units, each comprising one or more parameters of the artificial intelligence model, based on the structural information; storing the plurality of data units in memory; and determining whether to cache each of the plurality of data units in a cache. Effects of the invention

[0009] According to embodiments of the present disclosure, a storage device and a method of operation thereof can be provided that can improve read performance for an artificial intelligence model stored in a storage device.

[0010] The effects of the embodiments of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description in the claims. Brief explanation of the drawing

[0011] The content of this disclosure will be more fully understood from the detailed description and accompanying drawings provided below, which are provided solely for illustrative purposes and are not intended to limit the content of this disclosure. FIG. 1 is a schematic diagram of a storage device according to the present disclosure. Figure 2 is a block diagram schematically showing the memory of Figure 1. FIG. 3 is a diagram showing the schematic operation of a storage device according to the present disclosure. FIG. 4 is a diagram showing an example of structural information for an artificial intelligence model according to the present disclosure. FIG. 5 is a diagram illustrating an example of an operation in which a storage device according to the present disclosure determines a plurality of data units. FIG. 6 is a diagram showing an example of the structure of a plurality of data units according to the present disclosure. FIG. 7 is a flowchart illustrating the operation of a storage device according to the present disclosure determining whether to cache a first data unit in a cache. FIG. 8 is a diagram illustrating an example of an operation in which a storage device according to the present disclosure caches data units in a cache. FIG. 9 is a diagram showing another example of an operation in which a storage device according to the present disclosure caches data units in a cache. FIG. 10 is a diagram illustrating an example of an operation in which a storage device according to the present disclosure extracts a data unit from a cache. FIG. 11 is a diagram illustrating a method of operation of a storage device according to the present disclosure. Specific details for implementing the invention

[0012] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the exemplary drawings. In assigning reference numerals to the components of each drawing, the same components may have the same reference numeral as much as possible, even if they are shown in different drawings. Furthermore, in describing the present disclosure, if it is determined that a detailed description of related known components or functions may obscure the essence of the present disclosure, such detailed description may be omitted. Where terms such as "comprising," "having," or "consisting of" are used in this specification, other parts may be added unless "only" is used. Where a component is expressed in the singular, it may include a plural unless there is a special explicit description otherwise.

[0013] Additionally, terms such as first, second, A, B, (a), (b), etc., may be used to describe the components of the present disclosure. These terms are used merely to distinguish the components from other components, and the nature, order, sequence, or number of the components are not limited by such terms.

[0014] In describing the positional relationship of components, where it is stated that two or more components are "connected," "combined," or "joined," it should be understood that while the two or more components may be directly "connected," "combined," or "joined," they may also be "connected," "combined," or "joined" with other components "intervened." Here, the other components may be included in one or more of the two or more components that are "connected," "combined," or "joined" with one another.

[0015] In describing the temporal flow relationship regarding components, methods of operation, or methods of production, for example, when the temporal or sequential relationship is described using "after," "following," "next," or "before," it may include cases where the relationship is not continuous unless "immediately" or "directly" is used.

[0016] Meanwhile, where numerical values ​​or corresponding information regarding a component (e.g., levels, etc.) are mentioned, even without separate explicit notation, the numerical values ​​or corresponding information may be interpreted as including a range of error that may occur due to various factors (e.g., process factors, internal or external shocks, noise, etc.).

[0017] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0018] FIG. 1 is a schematic diagram of a storage device according to the present disclosure.

[0019] Referring to FIG. 1, a storage device (100) according to embodiments of the present disclosure may include a memory (110) for storing data and a controller (120) for controlling the memory (110).

[0020] The memory (110) includes a plurality of memory blocks and operates in response to the control of the controller (120). Here, the operation of the memory (110) may include, for example, a read operation, a program operation (also called a "write operation"), and an erase operation.

[0021] The memory (110) may include a memory cell array comprising a plurality of memory cells (also simply called "cells") that store data. Such a memory cell array may exist within a memory block.

[0022] For example, the memory (110) can be implemented in various types such as NAND Flash Memory, 3D NAND Flash Memory, NOR Flash Memory, Resistive Random Access Memory (RRAM), Phase-Change Memory (PRAM), Magnetoresistive Random Access Memory (MRAM), Ferroelectric Random Access Memory (FRAM), or Spin Transfer Torque Random Access Memory (STT-RAM).

[0023] Meanwhile, the memory (110) can be implemented as a three-dimensional array structure. The embodiments of the present disclosure can be applied to flash memory in which the charge storage layer is composed of a conductive floating gate, as well as to charge trap flash (CTF) in which the charge storage layer is composed of an insulating film.

[0024] The memory (110) receives commands and addresses, etc. from the controller (120) and can access an area selected by the address among the memory cell array. That is, the memory (110) can perform an operation instructed by the command on the area selected by the address.

[0025] For example, the memory (110) can perform program operations, read operations, and erase operations. In this regard, when performing a program operation, the memory (110) can program data in an area selected by an address. When performing a read operation, the memory (110) can read data from an area selected by an address. When performing an erase operation, the memory (110) can erase data stored in an area selected by an address.

[0026] The controller (120) can control write (program), read, erase, and background operations for memory (110). Here, background operations may include one or more of, for example, garbage collection (GC), wear leveling (WL), read reclaim (RR), or bad block management (BBM) operations.

[0027] The controller (120) can control the operation of the memory (110) in response to a request from a device (e.g., a host) located outside the storage device (100). On the other hand, the controller (120) can also control the operation of the memory (110) independently of the request from the host.

[0028] A HOST is a computer, UMPC (Ultra Mobile PC), workstation, PDA (Personal Digital Assistants), tablet, mobile phone, smartphone, e-book, PMP (portable multimedia player), portable game console, navigation device, black box, digital camera, DMB (Digital Multimedia Broadcasting) player, smart television, digital audio recorder, digital audio player, digital picture recorder, digital picture player, digital video recorder, digital video player, storage constituting a data center, one of the various electronic devices constituting a home network, one of the various electronic devices constituting a computer network, one of the various electronic devices constituting a telematics network, RFID (radio frequency identification) device, and a mobile device capable of driving under human control or autonomous driving (e.g. It may be a vehicle, robot, drone, etc. Alternatively, the host may be a virtual reality (VR) device that provides two-dimensional or three-dimensional virtual reality images or an augmented reality (AR) device that provides augmented reality images. In addition, the host may be any one of various electronic devices that require a storage device (100) capable of storing data.

[0029] The host may include at least one operating system (OS). The operating system can manage and control the overall functions and operations of the host and control the interaction between the host and the storage device (100). The operating system can be classified into a general operating system and a mobile operating system depending on the mobility of the host.

[0030] Meanwhile, the controller (120) and the host (HOST) may be separate devices. In some cases, the controller (120) and the host (HOST) may be implemented as a single integrated device. For convenience of explanation, the following description uses an example where the controller (120) and the host (HOST) are separate devices.

[0031] Referring to FIG. 1, the controller (120) may include a memory interface (122) and a control circuit (123), and may further include a host interface (121), etc.

[0032] The host interface (121) provides an interface for communication with a host. For example, the host interface (121) provides an interface that uses at least one of various interface protocols, such as USB (Universal Serial Bus) protocol, MMC (multimedia card) protocol, PCI (peripheral component interconnection) protocol, PCI-E (PCI-express) protocol, ATA (Advanced Technology Attachment) protocol, Serial-ATA protocol, Parallel-ATA protocol, SCSI (small computer small interface) protocol, ESDI (enhanced small disk interface) protocol, SMBus (System Management Bus) protocol, I2C (Inter-Integrated Circuit) protocol, I3C (Improved Inter-Integrated Circuit) protocol, and IDE (Integrated Drive Electronics) protocol, private protocol, etc.

[0033] When the control circuit (123) receives a command from the host (HOST), it can receive the command through the host interface (121) and perform an operation to process the received command.

[0034] The memory interface (122) can be connected to the memory (110) and provide an interface for communication with the memory (110). That is, the memory interface (122) can be configured to provide an interface between the memory (110) and the controller (120) in response to the control of the control circuit (123).

[0035] The control circuit (123) controls the operation of the memory (110) by performing overall control operations of the controller (120). To this end, for example, the control circuit (123) may include one or more of a processor (124), a working memory (125), etc., and may optionally include an error detection and correction circuit (ECC Circuit, 126), etc.

[0036] The processor (124) can control the general operations of the controller (120) and perform logical operations. The processor (124) can communicate with the host (HOST) through the host interface (121) and with the memory (110) through the memory interface (122).

[0037] The processor (124) can perform logical operations necessary to perform the functions of the Flash Translation Layer (FTL). The processor (124) can perform logical operations to convert a logical block address (LBA) provided by the host into a physical block address (PBA) through the Flash Translation Layer (FTL). The Flash Translation Layer (FTL) can receive a logical block address (LBA) as input using a mapping table and convert it into a physical block address (PBA).

[0038] There are various address mapping methods in the Flash Translation Layer depending on the mapping unit. Representative address mapping methods include the Page mapping method, the Block mapping method, and the Hybrid mapping method.

[0039] The processor (124) can render data received from the host (HOST). For example, the processor (124) can render data received from the host (HOST) using a set rendering seed. The rendered data is provided to memory (110) and can be programmed into a memory cell array of memory (110).

[0040] The processor (124) can deranderize data received from memory (110) during a read operation. For example, the processor (124) can deranderize data received from memory (110) using a deranderizing seed. The deranderized data can be output to a host (HOST).

[0041] The processor (124) can control the operation of the controller (120) by executing firmware. In other words, the processor (124) can execute (drive) the firmware loaded into the working memory (125) at boot time to control the general operation of the controller (120) and perform logical operations. The operation of the storage device (100) described below in the embodiments of the present disclosure can be implemented in such a way that the processor (124) executes the firmware in which the operation is defined.

[0042] Firmware is a program executed within a storage device (100) to operate the storage device (100), and may include various functional layers. For example, the firmware may include binary data in which code for executing each of the aforementioned functional layers is defined.

[0043] For example, the firmware may include one or more of the following: a Flash Translation Layer (FTL) that performs a conversion function between a logical address requested by a host (HOST) to a storage device (100) and a physical address of memory (110); a Host Interface Layer (HIL) that interprets a command requested by a host (HOST) to a storage device (100) and transmits it to the Flash Translation Layer (FTL); and a Flash Interface Layer (FIL) that transmits a command directed by the Flash Translation Layer (FTL) to memory (110).

[0044] This firmware may be loaded into working memory (125) from, for example, memory (110) or a separate non-volatile memory (e.g., ROM, NOR Flash) located outside of memory (110). When the processor (124) executes a boot operation after power-on, it may first load all or part of the firmware into working memory (125).

[0045] The processor (124) can perform logical operations defined in the firmware loaded in the working memory (125) to control the overall operation of the controller (120). The processor (124) can store the result of performing the logical operations defined in the firmware in the working memory (125). The processor (124) can control the controller (120) to generate a command or signal according to the result of performing the logical operations defined in the firmware. If the part of the firmware where the logical operations to be performed are defined is stored in the memory (110) but is not loaded in the working memory (125), the processor (124) can generate an event (e.g., an interrupt) to load the corresponding part of the firmware from the memory (110) to the working memory (125).

[0046] Meanwhile, the processor (124) can load metadata required to run the firmware from the memory (110). The metadata is data for managing the memory (110) and may include management information for user data stored in the memory (110).

[0047] Meanwhile, the firmware may be updated while the storage device (100) is being produced or while the storage device (100) is running. The controller (120) may download new firmware from outside the storage device (100) and update the existing firmware to the new firmware.

[0048] The working memory (125) can store firmware, program code, commands, or data required to drive the controller (120). This working memory (125) may include, for example, one or more of volatile memory such as SRAM (Static RAM), DRAM (Dynamic RAM), and SDRAM (Synchronous DRAM). Meanwhile, in addition to the working memory (125), the controller (120) may additionally use a separate volatile memory (e.g., SRAM, DRAM, SDRAM) located outside the controller (120).

[0049] The error detection and correction circuit (126) can detect an error bit of the target data using an error correction code and correct the detected error bit. Here, the target data may be, for example, data stored in the working memory (125) or data read from the memory (110).

[0050] The error detection and correction circuit (126) can be implemented to decode data into an error correction code. The error detection and correction circuit (126) can be implemented with various code decoders. For example, a decoder that performs unsystematic code decoding or a decoder that performs systematic code decoding may be used.

[0051] For example, the error detection and correction circuit (126) can detect error bits in sector units set for each of the read data. That is, each read data can be composed of multiple sectors. A sector can refer to a data unit smaller than a page, which is the read unit of the flash memory. The sectors constituting each read data can correspond to each other via an address.

[0052] The error detection and correction circuit (126) can calculate the bit error rate (BER) and determine whether correction is possible on a sector basis. For example, the error detection and correction circuit (126) can determine that a sector is uncorrectable (or fail) if the bit error rate (BER) is higher than a set reference value. On the other hand, if the bit error rate (BER) is lower than the reference value, the sector can be determined to be correctable (or pass).

[0053] The error detection and correction circuit (126) can sequentially perform error detection and correction operations for all read data. If a sector included in the read data is correctable, the error detection and correction circuit (126) may omit the error detection and correction operation for that sector for the next read data. When the error detection and correction operation for all read data is completed, the error detection and correction circuit (126) can detect the sector determined to be uncorrectable until the very end. There may be one or more sectors determined to be uncorrectable. The error detection and correction circuit (126) can transmit information (e.g., address information) about the sector determined to be uncorrectable to the processor (124).

[0054] The bus (127) may be configured to provide a channel between the components (121, 122, 124, 125, 126) of the controller (120). This bus (127) may include, for example, a control bus for transmitting various control signals, commands, etc., and a data bus for transmitting various data.

[0055] Meanwhile, some of the aforementioned components (121, 122, 124, 125, 126) of the controller (120) may be deleted, or some of the aforementioned components (121, 122, 124, 125, 126) of the controller (120) may be integrated into one. In some cases, one or more other components may be added in addition to the aforementioned components of the controller (120).

[0056] Below, the memory (110) will be described in more detail with reference to FIG. 2.

[0057] Figure 2 is a block diagram schematically showing the memory of Figure 1.

[0058] Referring to FIG. 2, a memory (110) according to embodiments of the present disclosure may include a memory cell array (210), an address decoder (220), a read and write circuit (230), control logic (240), and a voltage generation circuit (250), etc.

[0059] The memory cell array (210) may include a plurality of memory blocks (BLK1 to BLKz, where z is a natural number greater than or equal to 2).

[0060] In a plurality of memory blocks (BLK1~BLKz), a plurality of word lines (WL) and a plurality of bit lines (BL) are arranged, and a plurality of memory cells (MC) can be arranged.

[0061] Multiple memory blocks (BLK1 to BLKz) can be connected to an address decoder (220) through multiple word lines (WL). Multiple memory blocks (BLK1 to BLKz) can be connected to a read and write circuit (230) through multiple bit lines (BL).

[0062] Each of the plurality of memory blocks (BLK1~BLKz) may include a plurality of memory cells. For example, the plurality of memory cells may be non-volatile memory cells and may be composed of non-volatile memory cells having a vertical channel structure.

[0063] The memory cell array (210) may be configured as a memory cell array with a two-dimensional structure, and in some cases, may be configured as a memory cell array with a three-dimensional structure.

[0064] Meanwhile, each of the plurality of memory cells included in the memory cell array (210) can store at least 1 bit of data. For example, each of the plurality of memory cells included in the memory cell array (210) may be a Single-Level Cell (SLC) that stores 1 bit of data. For another example, each of the plurality of memory cells included in the memory cell array (210) may be a Multi-Level Cell (MLC) that stores 2 bits of data. For yet another example, each of the plurality of memory cells included in the memory cell array (210) may be a Triple-Level Cell (TLC) that stores 3 bits of data. For yet another example, each of the plurality of memory cells included in the memory cell array (210) may be a Quad-Level Cell (QLC) that stores 4 bits of data. For yet another example, the memory cell array (210) may include a plurality of memory cells that each store 5 bits or more of data.

[0065] At this time, the number of bits of data stored in each of the multiple memory cells can be dynamically determined. For example, a single-level cell storing 1 bit of data can be changed to a triple-level cell storing 3 bits of data.

[0066] Referring to FIG. 2, an address decoder (220), a read and write circuit (230), a control logic (240), and a voltage generation circuit (250), etc., can operate as peripheral circuits that drive the memory cell array (210).

[0067] The address decoder (220) can be connected to the memory cell array (210) through a plurality of word lines (WL).

[0068] The address decoder (220) can be configured to operate in response to the control of the control logic (240).

[0069] The address decoder (220) can receive an address through an input / output buffer inside the memory (110). The address decoder (220) can be configured to decode a block address among the received addresses. The address decoder (220) can select at least one memory block according to the decoded block address.

[0070] The address decoder (220) can receive a read voltage (Vread) and a pass voltage (Vpass) from the voltage generation circuit (250).

[0071] The address decoder (220) can apply a read voltage (Vread) to a selected word line (WL) within a selected memory block and apply a pass voltage (Vpass) to the remaining unselected word lines (WL) during a read operation.

[0072] When the address decoder (220) performs a program verification operation, it can apply a verification voltage generated by the voltage generation circuit (250) to a selected word line (WL) within a selected memory block and apply a pass voltage (Vpass) to the remaining unselected word lines (WL).

[0073] The address decoder (220) may be configured to decode column addresses among the received addresses. The address decoder (220) may transmit the decoded column addresses to the read and write circuit (230).

[0074] The read operation and program operation of the memory (110) can be performed in page units. The address received when requesting the read operation and program operation may include one or more of a block address, a row address, and a column address.

[0075] The address decoder (220) can select one memory block and one word line according to the block address and row address. The column address can be decoded by the address decoder (220) and provided to the read and write circuit (230).

[0076] The address decoder (220) may include one or more of a block decoder, a row decoder, a column decoder, and an address buffer.

[0077] The read and write circuit (230) may include a plurality of page buffers (PB). The read and write circuit (230) may operate as a "read circuit" during a read operation of the memory cell array (210) and as a "write circuit" during a write operation.

[0078] The above-described read and write circuit (230) is also referred to as a page buffer circuit or data register circuit that includes a plurality of page buffers (PB). Here, the read and write circuit (230) may include a data buffer responsible for data processing functions, and, in some cases, may additionally include a cache buffer responsible for caching functions.

[0079] Multiple page buffers (PB) can be connected to a memory cell array (210) via multiple bit lines (BL). During read operations and program verification operations, the multiple page buffers (PB) can continuously supply a sensing current to the bit lines (BL) connected to the memory cells to sense the threshold voltage (Vth) of the memory cells, and can detect through a sensing node that the amount of current flowing changes according to the program state of the corresponding memory cell and latch it as sensing data.

[0080] The read and write circuit (230) can operate in response to page buffer control signals output from the control logic (240).

[0081] The read and write circuit (230) senses the data of the memory cell during a read operation, temporarily stores the read data, and then outputs the data (DATA) to the input / output buffer of the memory (110). As an exemplary embodiment, the read and write circuit (230) may include a column selection circuit, etc., in addition to page buffers (PB) or page registers.

[0082] The control logic (240) can be connected to an address decoder (220), a read and write circuit (230), and a voltage generation circuit (250), etc. The control logic (240) can receive a command (CMD) and a control signal (CTRL) through the input / output buffer of the memory (110).

[0083] The control logic (240) may be configured to control the general operation of the memory (110) in response to a control signal (CTRL). The control logic (240) may output a control signal to adjust the precharge potential level of the sensing node of a plurality of page buffers (PB).

[0084] The control logic (240) can control the read and write circuit (230) to perform a read operation of the memory cell array (210). The voltage generation circuit (250) can generate a read voltage (Vread) and a pass voltage (Vpass) used during a read operation in response to a voltage generation circuit control signal output from the control logic (240).

[0085] Meanwhile, each memory block of the aforementioned memory (110) may be composed of a plurality of pages corresponding to a plurality of word lines (WL) and a plurality of strings corresponding to a plurality of bit lines (BL).

[0086] In a memory block (BLK), multiple word lines (WL) and multiple bit lines (BL) may be arranged in an alternating manner. For example, each of the multiple word lines (WL) may be arranged in a row direction, and each of the multiple bit lines (BL) may be arranged in a column direction. As another example, each of the multiple word lines (WL) may be arranged in a column direction, and each of the multiple bit lines (BL) may be arranged in a row direction.

[0087] A memory cell connected to one of a plurality of word lines (WL) and one of a plurality of bit lines (BL) can be defined. A transistor can be placed in each memory cell.

[0088] For example, a transistor placed in a memory cell (MC) may include a drain, a source, and a gate, etc. The drain (or source) of the transistor may be connected directly to the corresponding bit line (BL) or via another transistor. The source (or drain) of the transistor may be connected directly to a source line (which may be ground) or via another transistor. The gate of the transistor may include a floating gate surrounded by an insulator and a control gate to which a gate voltage is applied from a word line (WL).

[0089] In each memory block, a first selection line (also called a source selection line or drain selection line) may be further disposed outside the first outermost word line that is more adjacent to the read and write circuit (230) among the two outermost word lines, and a second selection line (also called a drain selection line or source selection line) may be further disposed outside the other second outermost word line.

[0090] In some cases, one or more dummy word lines may be placed between the first outermost word line and the first selection line. Additionally, one or more dummy word lines may be placed between the second outermost word line and the second selection line.

[0091] The read operation and program operation (write operation) of the aforementioned memory block can be performed on a page basis, and the erase operation can be performed on a memory block basis.

[0092] FIG. 3 is a diagram showing the schematic operation of a storage device (100) according to the present disclosure.

[0093] Referring to FIG. 3, the storage device (100) may include memory (110) and a controller (120).

[0094] The memory (110) can store multiple data units (DU). Each of the multiple data units (DU) may include one or more parameters (P) of an artificial intelligence model (AI_MDL).

[0095] An artificial intelligence model (AI_MDL) can analyze input data and output inference results for the input data. An artificial intelligence model (AI_MDL) may include one or more layers (e.g., input layer, hidden layer, output layer) used to calculate inference results from input data. Parameters (P) of the artificial intelligence model (AI_MDL) may be placed in one or more layers.

[0096] An artificial intelligence model (AI_MDL) can be trained using training data. Training an artificial intelligence model (AI_MDL) means adjusting its parameters so that the output for each input is as similar as possible to the ground-truth.

[0097] Meanwhile, the learning process of an artificial intelligence model (AI_MDL) can be expressed as training and the result as learning; however, either training or learning may be used to describe the learning process or the result.

[0098] Artificial intelligence models (AI_MDL) can be implemented in various ways. For example, artificial intelligence models (AI_MDL) can be current or future machine learning models, such as models that have undergone algorithm-based machine learning (e.g., tree-based models, k-Nearest Neighbors, k-Means Clustering, Principal Component Analysis (PCA), support vector machine (SVM)) or models that have undergone artificial neural network-based learning (e.g., CNN, RNN).

[0099] The artificial intelligence model (AI_MDL) can be stored as a checkpoint during the training process and in the memory (110) in the form of an archive during the inference process. At this time, the artificial intelligence model (AI_MDL) can be stored sequentially in the memory (110).

[0100] Parameters included in the artificial intelligence model (AI_MDL) can be processed by multiple processors, and can be processed simultaneously or at similar times. In the learning and inference operations of the artificial intelligence model (AI_MDL), operations are performed in which multiple processors reference and compute the parameters of the artificial intelligence model (AI_MDL), and to this end, the storage device (100) can process multiple read requests for the artificial intelligence model (AI_MDL).

[0101] In this case, since multiple processors each read the parameters they use from the artificial intelligence model (AI_MDL), the workload for the artificial intelligence model (AI_MDL) may have the characteristics of a random read.

[0102] The controller (120) can receive structural information (STR_INFO) for an artificial intelligence model (AI_MDL) from outside the storage device (100) (e.g., host (HOST)).

[0103] In embodiments of the present disclosure, structure information (STR_INFO) may be determined in various forms. For example, structure information (STR_INFO) may include direct information regarding a plurality of data units (DU) (e.g., the number of a plurality of data units (DU), the size of each of a plurality of data units (DU), and a logical address area corresponding to each of a plurality of data units (DU)). For another example, structure information (STR_INFO) may include characteristic information used by the controller (120) to determine the structure of a plurality of data units (DU).

[0104] And the controller (120) can determine a plurality of data units (DU) based on the received structure information (STR_INFO). The controller (120) can determine, based on the structure information (STR_INFO), the structure in which the parameters (P) of the aforementioned artificial intelligence model (AI_MDL) are stored in memory (110).

[0105] And the controller (120) can decide whether to cache each of the multiple data units (DU) in the cache.

[0106] A cache is a volatile memory capable of caching multiple data units (DU), and data units cached in the cache can be read faster than data units stored in memory (110).

[0107] In embodiments of the present disclosure, the cache may be implemented in various ways. For example, the cache may be a working memory (125) included in the controller (120).

[0108] In FIG. 3, the case where the cache is included inside the controller (120) is described as an example, but the cache may be implemented as a separate volatile or non-volatile memory located outside the controller (120).

[0109] At this time, the controller (120) can independently determine whether to cache each of the multiple data units (DU) in the cache for each of the multiple data units (DU). That is, the controller (120) can determine whether to cache one of the multiple data units (DU) in the cache regardless of whether another data unit is cached in the cache.

[0110] FIG. 4 is a diagram showing an example of structural information (STR_INFO) for an artificial intelligence model (AI_MDL) according to the present disclosure.

[0111] In embodiments of the present disclosure, structure information (STR_INFO) may include at least one of the number of processors accessing the artificial intelligence model (AI_MDL) in parallel (NUM_PROC), the number of parameters of the artificial intelligence model (AI_MDL) (NUM_PARAM), the size of each parameter of the artificial intelligence model (AI_MDL) (PARAM_SIZE), and layer information of the artificial intelligence model (AI_MDL) (LAYER_INFO). In this case, the layer information of the artificial intelligence model (AI_MDL) (LAYER_INFO) may include information of a logical address region corresponding to each of the layers included in the artificial intelligence model (AI_MDL).

[0112] In FIG. 4, the number of processors (NUM_PROC) accessing the artificial intelligence model (AI_MDL) in parallel is N, the size (PARAM_SIZE) of each parameter of the artificial intelligence model (AI_MDL) is 2 bytes, and the number of parameters (NUM_PARAM) of the artificial intelligence model (AI_MDL) is k. Also, the layer information (LAYER_INFO) of the artificial intelligence model (AI_MDL) indicates that data corresponding to the logical address range x ~ x+n corresponds to a fully connected layer, and data corresponding to the logical address range y ~ y+m corresponds to a recursive layer.

[0113] The controller (120) of the storage device (100) can configure a plurality of data units (DU) based on the aforementioned structure information (STR_INFO). Based on the structure information (STR_INFO), the controller (120) can predict the expected behavior of processors outside the storage device (100) (e.g., host (HOST)) accessing parameters of the artificial intelligence model (AI_MDL), and can configure a plurality of data units (DU) based on this.

[0114] For example, the controller (120) can determine the number of multiple data units (DU) to be proportional to the number of processors (NUM_PROC).

[0115] As another example, the controller (120) can determine the number of multiple data units (DU) to be proportional to the product of the size (PARAM_SIZE) of each parameter of the artificial intelligence model (AI_MDL) and the number (NUM_PARAM) of the parameters of the artificial intelligence model (AI_MDL).

[0116] As another example, the controller (120) can determine the size of each of the multiple data units (DU) based on the size of each logical address area of ​​the layers included in the layer information (LAYER_INFO) of the artificial intelligence model (AI_MDL).

[0117] FIG. 5 is a diagram showing an example of an operation in which a storage device (100) according to the present disclosure determines a plurality of data units (DU).

[0118] Referring to Fig. 5, the artificial intelligence model (AI_MDL) may include T layers (L1, L2, … , LT) (T is a natural number).

[0119] Each of the T layers (L1, L2, …, LT) may include one or more parameters. In FIG. 5, layer (L1) may include parameters (P11, P12, …), layer (L2) may include parameters (P21, P22, …), and layer (LT) may include parameters (PT1, PT2, …).

[0120] At this time, the controller (120) of the storage device (100) can determine the structure of a plurality of data units (DU) corresponding to an artificial intelligence model (AI_MDL) as follows.

[0121] For example, the parameters included in each of the multiple data units (DU) may be parameters included in a specific layer among the T layers (L1, L2, … , LT) included in the artificial intelligence model (AI_MDL).

[0122] In this case, all parameters included in a single data unit (DU) can be included in the same layer. And the parameters included in a single layer can be stored in one or more of the multiple data units (DU).

[0123] Through this, when receiving a read command for a specific layer of an artificial intelligence model (AI_MDL) from outside the storage device (100) (e.g., host (HOST)), the controller (120) can improve read performance by minimizing the number of data units accessed to process the read command.

[0124] In FIG. 5, the first data unit (DU_1) among the plurality of data units (DU) may include parameters (P11, P12, …) included in layer (L1) among the layers of the artificial intelligence model (AI_MDL).

[0125] And among the multiple data units (DU), the second data unit (DU_2) may include parameters (P21, P22, …) included in the layer (L2).

[0126] Likewise, among the multiple data units (DU), the k-th data unit (DU_K) may include parameters (PT1, PT2, …) included in the layer (LT). The value of K may be greater than or equal to T.

[0127] FIG. 6 is a diagram showing an example of the structure of a plurality of data units (DU) according to the present disclosure.

[0128] Referring to Fig. 6, each of the K data units (DU_1, DU_2, DU_3, … , DU_K) (where K is a natural number greater than or equal to 2) can correspond to one of the T layers (L1, L2, L3, … , LT) of the artificial intelligence model (AI_MDL).

[0129] Each data unit can correspond to a single continuous logical address range. Since parameters included in the same data unit are likely to be executed simultaneously or in adjacent time zones, they can be configured to be included in a single continuous logical address range.

[0130] In FIG. 6, the data unit (DU_1) corresponds to a logical address range of 0 to 99, the data unit (DU_2) corresponds to a logical address range of 100 to 499, the data unit (DU_3) corresponds to a logical address range of 500 to 1199, and the data unit (DU_K) corresponds to a logical address range of n to (n+m).

[0131] At this time, each data unit may be stored in a single continuous physical address area. In order to reduce the overhead incurred in the process of searching for the location where the parameters are stored in memory (110) when the operation of reading parameters included in the same data unit from memory (110) is executed, the parameters included in the data unit may be stored sequentially in a continuous physical address area.

[0132] In FIG. 6, data unit (DU_1) can be stored in physical address range A to (A + 99), data unit (DU_2) can be stored in physical address range B to (B + 399), data unit (DU_3) can be stored in physical address range C to (C + 699), and data unit (DU_K) can be stored in physical address range T to (T + m).

[0133] In the above, the operation of the storage device (100) determining the structure of a plurality of data units (DU) has been described.

[0134] Hereinafter, an operation is described in which a storage device (100) according to embodiments of the present disclosure determines whether to cache each of a plurality of data units (DU) in a cache.

[0135] FIG. 7 is a flowchart illustrating the operation of a storage device (100) according to the present disclosure determining whether to cache a first data unit (DU_1) in a cache (CACHE).

[0136] Referring to FIG. 7, the controller (120) of the storage device (100) can count the number of read commands received from outside the storage device (100) (e.g., host (HOST)) requesting a read of the first data unit (DU_1) during a predetermined first time interval (S710).

[0137] Then, the controller (120) determines whether the number of read commands received is greater than or equal to the first threshold read command count (S720). At this time, the first threshold read command count may be a preset value.

[0138] The number of first threshold read commands can be determined in various ways. For example, the number of first threshold read commands may be a preset value. As another example, the number of first threshold read commands may be determined based on the average cache hit rate of the cache during a predetermined time interval. The controller (120) may determine the number of first threshold read commands as a value proportional to the average cache hit rate.

[0139] When the number of read commands received is greater than or equal to the first threshold read command number (S720-Y), the controller (120) may decide to cache the first data unit (DU_1) in the cache (S730). This is because there is a high probability that the first data unit (DU_1) will be read, and thus there is a high probability that reading performance will be significantly improved when the first data unit (DU_1) is cached in the cache.

[0140] On the other hand, when the number of read commands received is less than the first threshold read command count (S720-N), the controller (120) may decide not to cache the first data unit (DU_1) in the cache (S740).

[0141] FIG. 8 is a diagram showing an example of an operation in which a storage device (100) according to the present disclosure caches a data unit in a cache.

[0142] Referring to FIG. 8, the controller (120) of the storage device (100) can cache a first data unit (DU_1) among a plurality of data units (DU) in a cache. At this time, the entire first data unit (DU_1) can be cached in the cache.

[0143] The controller (120) can perform an operation of internally reading the first data unit (DU_1) to cache the first data unit (DU_1) in the cache (CACHE).

[0144] Subsequently, when the controller (120) processes a read command for the first data unit (DU_1), it may access the first data unit (DU_1) cached in the cache instead of accessing the memory (110). By doing so, the controller (120) can process the read command for the first data unit (DU_1) more quickly, thereby improving read performance.

[0145] FIG. 9 is a diagram showing another example of an operation in which a storage device (100) according to the present disclosure caches a data unit in a cache.

[0146] Referring to FIG. 9, the controller (120) of the storage device (100) can cache a first data unit (DU_1) and a second data unit (DU_2) in a cache. The controller (120) can independently decide whether to cache the first data unit (DU_1) in the cache and whether to cache the second data unit (DU_2) in the cache.

[0147] When the first data unit (DU_1) is cached in the cache, the controller (120) can access the first data unit (DU_1) cached in the cache when processing a read command for the first data unit (DU_1).

[0148] Likewise, when the second data unit (DU_2) is cached in the cache, the controller (120) can access the second data unit (DU_2) cached in the cache when processing a read command for the second data unit (DU_2).

[0149] Meanwhile, data units cached in the cache may subsequently be evicted from the cache. Since the size of the cache is finite, existing data units cached in the cache must be evicted to secure space for caching other data units. This is explained in detail below in FIG. 10.

[0150] FIG. 10 is a diagram showing an example of an operation in which a storage device (100) according to the present disclosure extracts a data unit from a cache.

[0151] Referring to FIG. 10, the controller (120) of the storage device (100) can extract a first data unit (DU_1) cached in the cache from the cache. After the first data unit (DU_1) is extracted from the cache, the controller (120) can read the first data unit (DU_1) from memory (110) when processing a read command for the first data unit (DU_1).

[0152] In embodiments of the present disclosure, the controller (120) may eject the first data unit (DU_1) from the cache when a specific condition is satisfied. By doing so, the controller (120) can secure the free space necessary to cache another data unit in the cache.

[0153] For example, the controller (120) can extract the first data unit (DU_1) cached in the cache from the cache when the number of times the first data unit (DU_1) cached in the cache is accessed during a predetermined second time interval is less than the threshold number of accesses.

[0154] As another example, the controller (120) can extract the first data unit (DU_1) cached in the cache from the cache when the number of read commands reading the first data unit (DU_1) among M (M is a natural number) read commands received from outside the storage device (100) (e.g., host (HOST)) is less than the second threshold number of read commands.

[0155] As another example, the controller (120) can eject the first data unit (DU_1) cached in the cache from the cache when the cache hit rate of the first data unit (DU_1) among the data units cached in the cache is the lowest.

[0156] As another example, the controller (120) can eject the first data unit (DU_1) cached in the cache from the cache when the time at which the first data unit (DU_1) was cached in the cache is the oldest among the data units cached in the cache.

[0157] Meanwhile, regardless of whether the first data unit (DU_1) is evicted from the cache, the second data unit (DU_2) may remain cached in the cache. This is because whether the first data unit (DU_1) is cached in the cache and whether the second data unit (DU_2) is cached in the cache are determined independently.

[0158] FIG. 11 is a drawing showing a method of operation (1100) of a storage device (100) according to the present disclosure.

[0159] Referring to FIG. 11, the operation method (1100) of the storage device (100) may include the step (S1110) of receiving structural information (STR_INFO) for an artificial intelligence model (AI_MDL) from outside the storage device (100).

[0160] For example, structural information (STR_INFO) may include at least one of the number of processors accessing the artificial intelligence model (AI_MDL) in parallel (NUM_PROC), the number of parameters of the artificial intelligence model (AI_MDL) (NUM_PARAM), the size of each parameter of the artificial intelligence model (AI_MDL) (PARAM_SIZE), and layer information of the artificial intelligence model (AI_MDL) (LAYER_INFO).

[0161] And the operation method (1100) of the storage device (100) may include a step (S1120) of determining a plurality of data units (DU) based on the structure information (STR_INFO) received in step S1110. Each of the plurality of data units (DU) may include one or more parameters of an artificial intelligence model (AI_MDL).

[0162] For example, among a plurality of data units (DU), the first data unit (DU_1) may include parameters included in the first layer among the layers of the artificial intelligence model (AI_MDL).

[0163] The first data unit (DU_1) may correspond to a single continuous logical address area. The first data unit (DU_1) may be stored in a single continuous physical address area of ​​memory (110).

[0164] And the method of operation (1100) of the storage device (100) may include the step (S1130) of storing a plurality of data units (DU) in memory (110).

[0165] And the operation method (1100) of the storage device (100) may include a step (S1140) of determining whether to cache each of the plurality of data units (DU) in a cache.

[0166] For example, step S1140 may decide to cache the first data unit (DU_1) in a cache when the number of read commands received from outside the storage device (100) requesting to read the first data unit (DU_1) among a plurality of data units (DU) during a predetermined first time interval is greater than or equal to the number of first threshold read commands.

[0167] For example, the operation method (1100) of the storage device (100) may further include the step of ejecting the first data unit (DU_1) cached in the cache from the cache when the number of times the first data unit (DU_1) cached in the cache is accessed during a predetermined second time interval is less than the threshold number of accesses.

[0168] As another example, the operation method (1100) of the storage device (100) may further include the step of ejecting the first data unit (DU_1) cached in the cache from the cache when the number of read commands reading the first data unit (DU_1) among M (M is a natural number) read commands received from outside the storage device (100) is less than the number of second threshold read commands.

[0169] The foregoing description is merely an illustrative explanation of the technical concept of the present disclosure, and those skilled in the art to which the present disclosure pertains may make various modifications and variations within the scope of the essential characteristics of the present disclosure. Furthermore, the embodiments disclosed in the present disclosure are intended to explain, not limit, the technical concept of the present disclosure, and thus the scope of the technical concept of the present disclosure is not limited by these embodiments.

Claims

Claim 1 A storage device comprising: a memory for storing a plurality of data units, each including one or more parameters of an artificial intelligence model; and a controller that receives structural information for the artificial intelligence model from outside the storage device, determines the plurality of data units based on the structural information, and determines whether to cache each of the plurality of data units in a cache. Claim 2 In claim 1, the structure information comprises at least one of the number of processors accessing the artificial intelligence model in parallel, the number of parameters of the artificial intelligence model, the size of each of the parameters of the artificial intelligence model, and layer information of the artificial intelligence model. Claim 3 In claim 1, the first data unit among the plurality of data units is a storage device including parameters included in the first layer among the layers of the artificial intelligence model. Claim 4 In paragraph 3, the first data unit is a storage device corresponding to a single continuous logical address area. Claim 5 In paragraph 4, the first data unit is a storage device stored in one contiguous physical address area of ​​the memory. Claim 6 A storage device according to claim 1, wherein the controller determines to cache the first data unit in the cache when the number of read commands received from outside the storage device requesting to read the first data unit among the plurality of data units during a predetermined first time interval is greater than or equal to the number of first threshold read commands. Claim 7 In claim 6, the controller is a storage device that extracts a first data unit cached in the cache from the cache when the number of times the first data unit cached in the cache is accessed during a predetermined second time interval is less than a threshold number of accesses. Claim 8 In paragraph 6, the controller extracts the first data unit cached in the cache from the cache when the number of read commands reading the first data unit among M read commands received from outside the storage device is less than the second threshold number of read commands, and the storage device is a natural number M. Claim 9 A method of operating a storage device comprising: receiving structural information for an artificial intelligence model from outside the storage device; determining a plurality of data units, each comprising one or more parameters of the artificial intelligence model, based on the structural information; storing the plurality of data units in memory; and determining whether to cache each of the plurality of data units in a cache. Claim 10 In claim 9, the method of operation of a storage device comprising at least one of the structural information, the number of processors accessing the artificial intelligence model in parallel, the number of parameters of the artificial intelligence model, the size of each of the parameters of the artificial intelligence model, and layer information of the artificial intelligence model. Claim 11 In claim 9, the first data unit among the plurality of data units is a method of operation of a storage device including parameters included in the first layer among the layers of the artificial intelligence model. Claim 12 In paragraph 11, the above first data unit is a method of operation of a storage device corresponding to a single continuous logical address area. Claim 13 In paragraph 12, the above first data unit is a method of operation of a storage device stored in one contiguous physical address area of ​​the memory. Claim 14 In claim 9, the step of determining whether to cache each of the plurality of data units in the cache is a method of operation of a storage device that determines to cache the first data unit in the cache when the number of read commands received from outside the storage device requesting to read the first data unit among the plurality of data units during a predetermined first time interval is greater than or equal to the number of first threshold read commands. Claim 15 A method of operation of a storage device according to claim 14, further comprising the step of ejecting a first data unit cached in the cache from the cache when the number of times the first data unit cached in the cache is accessed during a predetermined second time interval is less than a threshold number of accesses. Claim 16 A method of operation of a storage device according to claim 14, further comprising the step of extracting a first data unit cached in the cache from the cache when the number of read commands reading the first data unit among M read commands received from outside the storage device is less than a second threshold number of read commands, wherein M is a natural number.

Images