Host and computing system including thereof

Abstract

The present invention relates to an electronic device. A computing system according to the present invention, capable of reducing the amount of data movement during embedding operations and efficiently processing embedding operations, comprises a host that requests a generation operation of a second embedding table composed of the first feature table group and a second feature table group, wherein the host divides the plurality of feature tables, each containing a plurality of embedding vectors for a plurality of elements, into a first feature table group and a second feature table group, generates a first embedding table composed of the first feature table group, and requests a generation operation of a second embedding table composed of the second feature table group, and a memory system that generates the second embedding table in accordance with the request for the generation operation provided by the host, wherein the host divides the plurality of feature tables into the first feature table group and the second feature table group based on the number of elements included in each of the plurality of feature tables.

Description

Technology Field

[0001] The present invention relates to an electronic device, and more specifically, to a host and a computing system including the same. Background Technology

[0002] A storage device is a device that stores data under the control of a host device, such as a computer or a smartphone. A storage device may include a memory device that stores data and a memory controller that controls the memory device. Memory devices can be classified into volatile memory devices and non-volatile memory devices.

[0003] Volatile memory devices may be memory devices that store data only while power is supplied and lose the stored data when the power supply is cut off. Volatile memory devices may include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), etc.

[0004] Non-volatile memory devices are memory devices in which data is not lost even when the power is cut off, and include ROM (Read Only Memory; ROM), PROM (Programmable ROM), EPROM (Electrically Programmable ROM), EEPROM (Electrically Erasable and Programmable ROM), and Flash Memory. Prior art literature

[65535] Published Patent Application No. 10-2015-0019359 (February 25, 2015) The problem to be solved

[0005] An embodiment of the present invention provides a host capable of reducing the amount of data movement during embedding operations and efficiently processing embedding operations, and a computing system including the same. means of solving the problem

[0006] A computing system according to an embodiment of the present invention comprises a host that requests a generation operation of a second embedding table composed of the first feature table group and a second feature table group, each comprising a plurality of embedding vectors for a plurality of elements, and a memory system that generates the second embedding table in accordance with the request for the generation operation provided by the host, wherein the host divides the plurality of feature tables into the first feature table group and the second feature table group based on the number of elements included in each of the plurality of feature tables.

[0007] A host according to an embodiment of the present invention comprises an embedding table division unit that divides a plurality of feature tables, each containing a plurality of embedding vectors for a plurality of elements, into a first feature table group and a second feature table group; an embedding table generation unit that generates a first embedding table composed of the first feature table group; and an embedding operation control unit that requests a memory system to generate a second embedding table composed of the second feature table group, wherein the embedding table division unit divides the plurality of feature tables into the first feature table group and the second feature table group based on the number of elements included in each of the plurality of feature tables.

[0008] A computing system according to an embodiment of the present invention comprises an embedding table partitioning unit that divides a plurality of feature tables, each containing a plurality of embedding vectors for a plurality of elements, into a plurality of feature table groups, and a plurality of computation processing units that generate an embedding table composed of one of the plurality of feature table groups, wherein the embedding table partitioning unit divides the plurality of feature tables into the plurality of feature table groups based on the number of elements included in each of the plurality of feature tables. Effects of the invention

[0009] According to the present technology, a host capable of reducing the amount of data movement during embedding operations and efficiently processing embedding operations, and a computing system including the same are provided. Brief explanation of the drawing

[0010] FIG. 1 is a drawing for explaining an example of a computing system according to an embodiment of the present invention. FIG. 2 is a drawing for explaining another example of a computing system according to one embodiment of the present invention. FIG. 3 is a drawing for explaining an embedding table according to an embodiment of the present invention. FIG. 4 is a drawing for explaining a feature table according to an embodiment of the present invention. FIG. 5 is a drawing for explaining a host according to an embodiment of the present invention. FIG. 6 is a drawing for explaining a memory system according to one embodiment of the present invention. FIG. 7 is a diagram illustrating the operation of dividing a plurality of feature tables according to one embodiment of the present invention. FIG. 8 is a diagram illustrating the operation of generating an embedding table according to an embodiment of the present invention. FIG. 9 is a diagram illustrating the operation of obtaining an embedding vector according to an embodiment of the present invention. FIG. 10 is a diagram illustrating the operation of updating an embedding table according to one embodiment of the present invention. FIG. 11 is a flowchart illustrating a communication method between a host and a memory system according to an embodiment of the present invention. FIG. 12 is a diagram illustrating a communication packet between a host and a memory system according to an embodiment of the present invention. FIG. 13 is a flowchart for explaining the operation method of a computing system according to one embodiment of the present invention. FIG. 14 is a flowchart illustrating a method for dividing a plurality of feature tables according to an embodiment of the present invention. Specific details for implementing the invention

[0011] Specific structural or functional descriptions regarding embodiments according to the concept of the present invention disclosed in this specification or application are provided merely for the purpose of explaining embodiments according to the concept of the present invention, and embodiments according to the concept of the present invention may be implemented in various forms and should not be interpreted as being limited to the embodiments described in this specification or application.

[0013] FIG. 1 is a drawing for explaining an example of a computing system according to an embodiment of the present invention.

[0014] Referring to FIG. 1, the computing system (10) may include a host (100) and a memory system (200).

[0015] The host (100) may include a host processor (110) and a host memory (120).

[0016] The host processor (110) may include circuits, interfaces, or program code for performing data processing and controlling the operations of components of the computing system (10). For example, the host processor (110) may include a general-purpose processor such as a CPU, AP, DSP (Digital Signal Processor), a graphics-dedicated processor such as a GPU, VPU (Vision Processing Unit), or an artificial intelligence-dedicated processor such as an NPU.

[0017] The host memory (120) may include SRAM or DRAM, etc., for storing data, instructions, or program codes necessary for the operation of the computing system (10). It may also include non-volatile memory. In one embodiment, the host memory (120) may store program codes capable of operating to run one or more operating systems (OS) and virtual machines (VMs).

[0018] The host processor (110) can execute one or more operating systems and virtual machines by running program codes stored in the host memory (120). In this way, the host processor (110) can control the operation of the components of the computing system (10).

[0019] The host (100) can communicate with the memory system (200) using at least one of various communication methods such as USB (Universal Serial Bus), SATA (Serial AT Attachment), SAS (Serial Attached SCSI), HSIC (High Speed ​​Interchip), SCSI (Small Computer System Interface), PCI (Peripheral Component Interconnection), PCIe (PCI express), NVMe (NonVolatile Memory express), UFS (Universal Flash Storage), SD (Secure Digital), MMC (MultiMedia Card), eMMC (embedded MMC), DIMM (Dual In-line Memory Module), RDIMM (Registered DIMM), LRDIMM (Load Reduced DIMM).

[0020] The memory system (200) may be a device that stores data under the control of the host (100).

[0021] The memory system (200) can be manufactured as any one of various types of memory systems depending on the communication method with the host (100). For example, the memory system (200) can be configured as any one of various types of storage devices such as an SSD, MMC, eMMC, RS-MMC, micro-MMC type multimedia card, SD, mini-SD, micro-SD type secure digital card, USB (universal storage bus) storage device, UFS (universal flash storage) device, PCMCIA (personal computer memory card international association) card type storage device, PCI (peripheral component interconnection) card type storage device, PCI-E (PCI express) card type storage device, CF (compact flash) card, smart media card, memory stick, etc.

[0022] The memory system (200) can be manufactured in any one of various types of package forms. For example, the memory system (200) can be manufactured in any one of various types of package forms such as POP (package on package), SIP (system in package), SOC (system on chip), MCP (multi-chip package), COB (chip on board), WFP (wafer-level fabricated package), WSP (wafer-level stack package), etc.

[0023] In one embodiment, the memory system (200) may be a pooled memory. The pooled memory may have a structure in which a plurality of memory devices are connected in parallel.

[0024] The memory system (200) may include a plurality of memory devices (210), a memory controller (220), and an arithmetic unit (230).

[0025] The memory device (210) can store data. The memory device (210) operates in response to the control of the memory controller (220). The memory device (210) may include a memory cell array (not shown) comprising a plurality of memory cells that store data.

[0026] The memory device (210) is configured to receive a command and an address from the memory controller (220) and to access an area selected by the address among the memory cell arrays. The memory device (210) can perform the operation instructed by the command on the area selected by the address. For example, the memory device (210) can perform a write operation (program operation), a read operation, and an erase operation. During the program operation, the memory device (210) will program data into the area selected by the address. During the read operation, the memory device (210) will read data from the area selected by the address. During the erase operation, the memory device (210) will erase the data stored in the area selected by the address.

[0027] The memory controller (220) can control the overall operation of the memory system (200).

[0028] When power is applied to the memory system (200), the memory controller (220) can execute firmware (FW).

[0029] The memory controller (220) can control the memory device (210) to perform a program operation, a read operation, or an erase operation, etc., according to a request from the host (100). When performing a program operation, the memory controller (220) can provide a write command, an address, and data to the memory device (210). When performing a read operation, the memory controller (220) can provide a read command and an address to the memory device (210). When performing an erase operation, the memory controller (220) can provide an erase command and an address to the memory device (210).

[0030] In an embodiment, the memory controller (220) may control at least two memory devices (210). In this case, the memory controller (220) may control the memory devices (210) according to an interleaving method to improve operational performance. The interleaving method may be a method of controlling the operation of at least two memory devices (210) so that it overlaps.

[0031] The operation unit (230) can perform mathematical operations such as addition and multiplication. For example, the operation unit (230) may include a computing unit for performing mathematical operations.

[0032] Meanwhile, in the embodiment illustrated in FIG. 1, the memory controller (220) and the operation unit (230) are depicted as separate and independent devices, but are not necessarily limited thereto. For example, the operation unit (230) may be implemented as a component of the memory controller (220).

[0033] In one embodiment, the computing system (10) may be a device for providing a recommendation system. In this case, the recommendation system may recommend items that the user is likely to be interested in (e.g., movies, music, news, books, products, etc.) based on user information. In one embodiment, the computing system (10) may provide a recommendation system using a deep learning-based recommendation model. In this case, the recommendation model may be a learning model that is learned using a plurality of learning data sets. For example, a deep learning-based recommendation model may include a plurality of neural networks, and the plurality of neural networks may be learned using a plurality of learning data sets. A neural network may include a plurality of layers. For example, a neural network may include an input layer, a hidden layer, and an output layer. Additionally, a neural network may include a plurality of hidden layers. A neural network including a plurality of hidden layers is called a Deep Neural Network, and the process of training a Deep Neural Network is called Deep Learning. Hereinafter, training a neural network can be understood as training the parameters of the neural network. Furthermore, a trained neural network can be understood as a neural network to which the trained parameters have been applied.

[0034] In one embodiment, the recommendation system of the computing system (10) may be provided under the control of the host (100). For example, the host (100) may include a host processor (110) and a host memory (120). The host processor (110) may be a general-purpose processor such as a CPU, AP, DSP (Digital Signal Processor), etc., a graphics-dedicated processor such as a GPU, VPU (Vision Processing Unit), etc., or an artificial intelligence-dedicated processor such as an NPU. The host memory (120) may store an operating system or an application, etc., for providing the recommendation system.

[0035] Since deep learning-based recommendation systems perform memory-intensive embedding operations, they may cause bandwidth issues, and because a large amount of service data is required, there may be a problem with insufficient capacity of the host memory (120). Therefore, to perform embedding efficiently, the computing system (10) can perform embedding using a memory system (200).

[0036] In one embodiment, the host (100) may control the memory system (200) to obtain an embedding vector for target data. For example, the host (100) may request an embedding vector from the memory system (200). In one embodiment, a plurality of memory devices (210) may store an embedding table containing a plurality of embedding vectors. The memory system (200) may provide the embedding vector requested by the host (100) based on the embedding table.

[0037] The host (100) can perform various operations to output a recommendation result according to a preset algorithm using an embedding vector.

[0038] Additionally, in one embodiment, the host (100) may store an embedding table in the host memory (120). The host (100) may obtain an embedding vector by performing an embedding operation based on the embedding table stored in the host memory (120).

[0039] In other words, the computing system (10) can divide the embedding table used for embedding operations and store the divided embedding tables in the host (100) and the memory system (200), respectively. For example, the computing system (10) can divide the embedding table into a first embedding table and a second embedding table, store the first embedding table in the host (100), and store the second embedding table in the memory system (200). Accordingly, the computing system (10) can perform distributed processing of embedding operations using the embedding tables stored in the host (100) and the memory system (200), respectively.

[0041] FIG. 2 is a drawing for explaining another example of a computing system according to one embodiment of the present invention.

[0042] Referring to FIG. 2, the computing system (20) may include a host (100) and a plurality of memory systems (200-1 to 200-n).

[0043] For example, the computing system (20) may represent a structure in which a host (100) is connected to a plurality of memory systems (200-1 to 200-n). In this case, the plurality of memory systems (200-1 to 200-n) may represent a plurality of memory systems (200) of FIG. 1.

[0044] The computing system (20) can divide the embedding table used for embedding operations and store the divided embedding tables in the host (100) and a plurality of memory systems (200-1 to 200-n), respectively.

[0045] In one embodiment, the computing system (20) may divide the embedding table into a first embedding table and a second embedding table, store the first embedding table in a host (100), and store the second embedding table in a plurality of memory systems (200-1 to 200-n).

[0046] Additionally, in one embodiment, the computing system (20) may divide the embedding table according to the number of hosts (100) and multiple memory systems (200-1 to 200-n), and store the divided embedding tables in each of the hosts (100) and multiple memory systems (200-1 to 200-n). For example, the host (100) and the multiple memory systems (200-1 to 200-n) may each be a single computational processing unit. In this case, the computing system (20) may divide the multiple feature tables included in the embedding table into multiple feature table groups. Subsequently, the multiple computational processing units may each generate an embedding table composed of any one of the multiple feature table groups. Here, the multiple computational processing units may include the host (100) and multiple memory systems (200-1 to 200-n). Meanwhile, the operation of the computing system (20) dividing the multiple feature tables will be explained in detail with reference to FIG. 7.

[0047] Accordingly, the computing system (10) can distribute the embedding operations using embedding tables stored in the host (100) and multiple memory systems (200-1 to 200-n).

[0049] FIG. 3 is a drawing for explaining an embedding table according to an embodiment of the present invention.

[0050] Referring to FIG. 3, an embedding table stored in a host (100) or memory system (200) is illustrated.

[0051] An embedding table may contain vector information obtained through embedding learning using multiple training data. For example, an embedding table may contain vector information regarding items of a previously trained training model. Here, the training model may refer to an artificial intelligence model, and the artificial intelligence model may be characterized as being created through learning. Specifically, the embedding table may be a collection of categorical data that can be classified into categories and corresponding embedding vector pairs, and the training data may each be categorical data. The categorical data may be quantified into vector forms with mutual similarity using an embedding algorithm. For example, a vector may be a collection of multiple integers or floating-point numbers, such as '(3, 5)' or '(0.1, -0.5, 2, 1.2)'. That is, an embedding table may refer to a set of categorical data from a training model trained to be classified by category and vector information of the categorical data. The more similar the vector values, such as the gradient of the embedding vector and the shape of the embedding vector, the more semantically similar the words corresponding to the embedding vectors may be.

[0052] Meanwhile, categories can be referred to as 'features,' and training data, categorical data, or items can be referred to as 'elements.'

[0053] According to one embodiment of the present invention, an embedding table may be configured in three dimensions. Specifically, referring to FIG. 3, the embedding table may be configured in three dimensions including 'Table sizes', '# of features', and 'dimension'. Here, 'Table sizes' may refer to the number of elements included in a specific feature. For example, if 'Harry Potter' and 'Shrek' are included in the feature <Movie Name>, there may be 2 elements. Also, '# of features' may refer to the number of feature types. For example, if the embedding table includes <Movie Name> and <Movie Genre>, there may be 2 feature types. Finally, 'dimension' may refer to the dimension of the embedding vector. That is, it may refer to the number of numbers included in the vector. Since '(0.1, -0.5, 2, 1.2)' includes 4 numbers, there may be 4 dimensions.

[0055] FIG. 4 is a drawing for explaining a feature table according to an embodiment of the present invention.

[0056] Referring to FIG. 4, the embedding table may be composed of a plurality of feature tables (features 1 to features n).

[0057] A plurality of feature tables (features 1 to features n) may each include a plurality of embedding vectors for a plurality of elements. Each feature table (features 1 to features n) may include embedding vectors for elements included in a corresponding feature. Specifically, each feature table (features 1 to features n) may include embedding vectors quantified based on the similarity between elements included in a corresponding feature. In one embodiment, the plurality of feature tables (features 1 to features n) may have different numbers of elements. For example, the first feature table (features 1) may include embedding vectors for 11 elements. The second feature table (features 2) may include embedding vectors for 4 elements. The third feature table (features 3) may include embedding vectors for 7 elements. The fourth feature table (features 4) may contain embedding vectors for two elements.

[0058] Meanwhile, the more elements included in a feature table, the fewer the number of lookups for each element included in that feature table. In this case, the elements included in that feature table may have low temporal locality. Conversely, the fewer elements included in a feature table, the more the number of lookups for each element included in that feature table may be. In this case, the elements included in that feature table may have high temporal locality. Therefore, if embedding operations are processed using a single processor without considering the temporal locality of the feature tables, inefficient memory access may occur. For example, when embedding operations are processed on a host (100), the CPU has a multi-level cache hierarchy structure, so it may be inefficient for the CPU to process feature tables containing elements with low temporal locality. Conversely, it may be efficient for feature tables containing elements with high temporal locality to be processed by the CPU of the host (100).

[0059] Accordingly, according to one embodiment of the present invention, by dividing the embedding table and generating the divided embedding tables in the host (100) and the memory system (200), an efficient embedding operation considering temporal locality can be performed.

[0061] FIG. 5 is a drawing for explaining a host according to an embodiment of the present invention.

[0062] Referring to FIG. 5, the host (100) may include an embedding table partitioning unit (111), a first embedding table management unit (112), an embedding operation control unit (113), and a host interface (114).

[0063] The embedding table partitioning unit (111) can partition the embedding table. For example, the embedding table partitioning unit (111) can partition a plurality of feature tables included in the embedding table into a first feature table group and a second feature table group.

[0064] In one embodiment, the embedding table division unit (111) can divide the plurality of feature tables into a first feature table group and a second feature table group based on the number of elements included in each of the plurality of feature tables.

[0065] The first embedding table management unit (112) can generate a first embedding table composed of a first feature table group. Additionally, the first embedding table management unit (112) can update the first embedding table under the control of the embedding operation control unit (113).

[0066] The embedding operation control unit (113) can control the embedding operation of the host (100) and the memory system (200). In one embodiment, the embedding operation may include an embedding table creation operation (or, an initialization operation), an inference operation, and a training operation. The inference operation may be an operation of looking up an embedding vector from an embedding table. The training operation may be an operation of calculating a gradient and updating an embedding table. Here, the gradient is data for updating the embedding table and may be data including weights.

[0067] In one embodiment, the embedding operation control unit (113) can control the first embedding table management unit (112) to create a first embedding table. Additionally, the embedding operation control unit (113) can request the memory system (200) to create a second embedding table composed of a second feature table group. For example, the embedding operation control unit (113) can request the creation of the second embedding table through the host interface (114).

[0068] In one embodiment, the embedding operation control unit (113) may obtain an embedding vector of a target element by performing an embedding operation based on a first embedding table, or request an embedding vector of a target element from a memory system (200).

[0069] In one embodiment, the embedding operation control unit (113) can update the first embedding table and the second embedding table based on weights obtained through embedding learning. For example, the embedding operation control unit (113) can control the first embedding table management unit (112) to update the first embedding table based on weights. Additionally, the embedding operation control unit (113) can request the memory system (200) to update the second embedding table based on weights.

[0071] FIG. 6 is a drawing for explaining a memory system according to one embodiment of the present invention.

[0072] The plurality of memory devices (210), memory controller (220), and operation unit (230) of FIG. 6 may represent the plurality of memory devices (210), memory controller (220), and operation unit (230) of FIG. 1.

[0073] In one embodiment, a plurality of memory devices (210) can store a second embedding table.

[0074] The memory controller (220) may include a memory interface (221) and a second embedding table management unit (222).

[0075] The memory interface (221) can relay packets between the host (100) and the memory system (200). For example, the memory interface (221) can transmit requests received from the host (100) to the second embedding table management unit (222) and the operation unit (230). The method of communication between the host (100) and the memory system (200) will be described in detail with reference to FIG. 11.

[0076] The second embedding table management unit (222) can generate a second embedding table. For example, the second embedding table management unit (222) can control a plurality of memory devices (210) to allocate a memory area to store the second embedding table.

[0077] In one embodiment, the second embedding table management unit (222) can update the second embedding table based on weights in response to an update request from the host (100).

[0078] In one embodiment, the operation unit (230) can perform an embedding operation based on the second embedding table. The embedding data obtained through the embedding operation can be provided to the host (100) through the memory interface (221).

[0080] FIG. 7 is a diagram illustrating the operation of dividing a plurality of feature tables according to one embodiment of the present invention.

[0081] In the graph shown in Fig. 7, the vertical axis represents the number of elements, and the horizontal axis represents the sequence number of feature tables.

[0082] In one embodiment, the embedding table partitioning unit (111) can generate a baseline based on the number of multiple feature tables and the number of all elements included in the multiple feature tables. Here, the baseline may be the result of an arithmetic mean using the number of multiple feature tables and the number of all elements. For example, the baseline may be calculated using the following mathematical formula 1.

[0083]

[0084] In mathematical formula 1, slope represents the baseline, # of Element represents the number of elements, and # of features represents the number of multiple feature tables.

[0085] In one embodiment, the embedding table partitioning unit (111) can sort the multiple feature tables in ascending order based on the number of elements included in each of the multiple feature tables. Additionally, the embedding table partitioning unit (111) can divide the multiple feature tables into a first feature table group and a second feature table group based on a reference line.

[0086] For example, the embedding table partitioning unit (111) can determine the feature tables located below the reference line among the plurality of feature tables as the first feature table group. Additionally, the embedding table partitioning unit (111) can determine the feature tables located above the reference line among the plurality of feature tables as the second feature table group.

[0087] Meanwhile, the operation of dividing multiple feature tables described with reference to FIG. 7 is merely an example. Therefore, the operation of dividing multiple feature tables may vary depending on the embodiment.

[0089] FIG. 8 is a diagram illustrating the operation of generating an embedding table according to an embodiment of the present invention.

[0090] Referring to FIG. 8, the embedding table splitting unit (111) can split a plurality of feature tables into a first feature table group and a second feature table group. Subsequently, the embedding table splitting unit (111) can provide information (table_info 1) about the first feature table group to the first embedding table management unit (112).

[0091] The first embedding table management unit (112) can generate a first embedding table based on information (table_info 1) about a first feature table group. For example, the first embedding table management unit (112) can control the host memory (120) to allocate a memory area to store the first embedding table. After allocating the memory area, the host memory (120) can store zero values, random values, etc., in the allocated memory area. Thus, the initial embedding table consists of dummy data such as zero values, random values, etc., but the embedding table can be updated through embedding learning.

[0092] Additionally, the embedding table partitioning unit (111) can provide information (table_info 2) about the second feature table group to the memory system (200) through the host interface (114). For example, the embedding operation control unit (113) can request the creation operation of the second embedding data to the memory system (200) through the host interface (114). The host interface (114) can provide the request for the creation operation and information (table_info 2) about the second feature table group to the memory interface (221). The memory interface (221) can provide information (table_info 2) about the second feature table group to the second embedding table management unit (222).

[0093] The second embedding table management unit (222) can create a second embedding table based on information (table_info 2) about the second feature table group. For example, the second embedding table management unit (222) can receive information about the size of the second embedding table from the host (100) in response to a request for a creation operation. The second embedding table management unit (222) can allocate a memory area to store the second embedding table based on information about the size of the second embedding table. For example, the second embedding table management unit (222) can control a plurality of memory devices (210) to allocate a memory area to store the second embedding table. After allocating the memory area, the plurality of memory devices (210) can store zero values, random values, etc., in the allocated memory area. Thus, the initial embedding table is composed of dummy data such as zero values, random values, etc., but the embedding table can be updated through embedding learning.

[0094] After creating the second embedding table, the memory interface (221) can provide a response corresponding to the request for a creation operation to the embedding operation control unit (113) through the host interface (114). Additionally, the memory interface (221) can provide the address of the allocated memory area to the embedding operation control unit (113) through the response.

[0096] FIG. 9 is a diagram illustrating the operation of obtaining an embedding vector according to an embodiment of the present invention.

[0097] Referring to FIG. 9, when a feature table containing a target element is included in a first feature table group, the embedding operation control unit (113) can perform an embedding operation based on the first embedding table (embedding table 1). At this time, the target element may represent an element that is the target for obtaining an embedding vector. For example, the embedding operation control unit (113) can perform an embedding operation by reading the first embedding table (embedding table 1) from the host memory (120). The embedding operation control unit (113) can obtain an embedding vector for the target element through the embedding operation. In one embodiment, the embedding operation may represent a multiplication operation between a one-hot vector corresponding to the target element and the embedding table. However, the embedding operation using a one-hot vector is merely one embodiment, and the embedding operation may be performed in various ways depending on the embodiment.

[0098] If a feature table containing a target element is included in a second feature table group, the embedding operation control unit (113) may request an embedding vector of the target element from the memory system (200). For example, the embedding operation control unit (113) may provide the request for the embedding vector and information related to the target element (target_info) to the memory interface (221) through the host interface (114). At this time, the information related to the target element (target_info) may include a one-hot vector corresponding to the target element, an address where the embedding vector of the target element is stored, etc.

[0099] The operation unit (230) can perform an embedding operation based on the second embedding table 2 in response to a request for an embedding vector of a target element. For example, the operation unit (230) can perform an embedding operation by reading the second embedding table 2 from a plurality of memory devices (210). The operation unit (230) can obtain an embedding vector of a target element through an embedding operation using information related to the target element (target_info) and the second embedding table 2. The operation unit (230) can provide the obtained embedding vector to the host (100) through the memory interface (221).

[0101] FIG. 10 is a diagram illustrating the operation of updating an embedding table according to one embodiment of the present invention.

[0102] Referring to FIG. 10, the embedding operation control unit (113) can update the first embedding table based on weights obtained through embedding learning. For example, the embedding operation control unit (113) can control the first embedding table management unit (112) to update the first embedding table based on weights. The first embedding table management unit (112) can update the first embedding table stored in the host memory (120) based on weights.

[0103] Additionally, the embedding operation control unit (113) can request the memory system (200) to update the second embedding table based on a weight. For example, the embedding operation control unit (113) can provide the request for the update operation and the weight to the memory interface (221) through the host interface (114).

[0104] The second embedding table management unit (222) can update the second embedding table stored in a plurality of memory devices (210) based on weights. After the update operation of the second embedding table is completed, the memory system (200) can provide a response corresponding to the request for the update operation to the embedding operation control unit (113) through the host interface (114).

[0106] FIG. 11 is a flowchart illustrating a communication method between a host and a memory system according to an embodiment of the present invention.

[0107] In one embodiment, to manage communication between the host (100) and the memory system (200), the memory interface (221) may include a mailbox and a Direct Memory Access (DMA) device.

[0108] In step S701, the host (100) may request a read of a message to the memory interface (221). At this time, the message may contain information for directing an embedding operation. In one embodiment, the embedding operation may be any one of an embedding table creation operation, an inference operation, and a learning operation. For example, when the host (100) completes the preparation of a query for an embedding operation, the host (100) may allocate host memory (120) and store the query. The host (100) may create a message based on the query and broadcast it to the memory system (200). For example, the host (100) may provide the message to a mailbox and request the memory system (200) to read the message.

[0109] In step S703, when a message is received from the host (100), the memory interface (221) can transmit an interrupt to the memory system (200).

[0110] In step S705, the memory system (200) can read a message from the memory interface (221). The memory system (200) can decode the message and allocate a memory area according to the size of the query or gradient.

[0111] In step S707, the memory system (200) can receive data stored in the host memory (120) through the memory interface (221) by referring to the source address included in the message. At this time, the data stored in the host memory (120) may be a query or a gradient. For example, the memory system (200) can receive data stored in the host memory (120) through a DMA device.

[0112] In step S709, the memory system (200) can perform an embedding operation.

[0113] In step S711, the memory system (200) can provide the result of the embedding operation to the host (100). For example, the memory system (200) can obtain the address of the host memory (120) to store the result from the message and provide the result to the address of the host memory (120).

[0114] In step S713, when a result is stored at an address of the host memory (120), the memory system (200) can respond via the memory interface (221). Specifically, the memory system (200) can provide a message corresponding to the response to a mailbox.

[0115] In step S715, when a message corresponding to the response is received, the memory interface (221) can transmit an interrupt to the host (100).

[0116] In step S717, the host (100) can read a message corresponding to the response from the memory interface (221).

[0118] FIG. 12 is a diagram illustrating a communication packet between a host and a memory system according to an embodiment of the present invention.

[0119] Referring to FIG. 12, a communication packet between a host (100) and a memory system (200) is illustrated. Specifically, the first communication packet (11) may be a message transmitted from the host (100) to the memory system (200). The first communication packet (11) may include a 'Task ID', 'Opcode', 'Source Address', 'Source Size', and 'Destination Address'.

[0120] The first communication packet (11) may consist of a total of 91 bits, and 4 bits may be allocated to the 'Task ID'. The 'Task ID' may indicate the operating status of the host (100). For example, the 'Task ID' may indicate whether the operation of the host (100) is running or terminated. The host (100) may use the 'Task ID' to reorder the operation of the memory system (200).

[0121] And, 3 bits are allocated to 'Opcode', and data for distinguishing multiple embedding operations may be included. Specifically, the host (100) may use 'Opcode' to distinguish an embedding table creation operation (or, initial setup operation), an inference operation, and a learning operation.

[0122] And, 32 bits are allocated to 'Source Address' and may contain information about the source address of a query or gradient. Specifically, the host (100) may use 'Source Address' to contain information about the address of a query or gradient that the memory system (200) must retrieve from the host memory (120).

[0123] And, 20 bits are allocated to 'Source Size' and may contain information about the size of the query or gradient. And, 'Destination Address' may contain the address of the host memory (120) to receive the result of the internal operation performed by the memory system (200).

[0124] A host (100) can communicate with a memory system (200) using a first communication packet (11). Then, when the memory system (200) receives the first communication packet (11) from the host (100), it can transmit a second communication packet (12), which is an acknowledgment message. Here, the second communication packet (12) may include a 'Task ID' and an 'Opcode'.

[0126] FIG. 13 is a flowchart for explaining the operation method of a computing system according to one embodiment of the present invention.

[0127] The method illustrated in FIG. 13 can be performed, for example, by the computing system (10) illustrated in FIG. 1 or the computing system (20) illustrated in FIG. 2. For convenience of explanation, the following description will be based on the computing system (10) illustrated in FIG. 1.

[0128] Referring to FIG. 13, in step S1301, the host (100) can divide a plurality of feature tables into a first feature table group and a second feature table group.

[0129] At this time, the host (100) can divide the multiple feature tables into a first feature table group and a second feature table group based on the number of elements included in each of the multiple feature tables.

[0130] In step S1303, the host (100) can generate a first embedding table composed of a first feature table group.

[0131] In step S1305, the host (100) may request the memory system (200) to create a second embedding table composed of a second feature table group.

[0132] In step S1307, the memory system (200) can create a second embedding table in response to a request for a creation operation of a second embedding table provided by the host (100).

[0133] At this time, the memory system (200) receives information about the size of the second embedding table from the host (100) and can allocate a memory area to store the second embedding table based on the information about the size.

[0135] FIG. 14 is a flowchart illustrating a method for dividing a plurality of feature tables according to an embodiment of the present invention.

[0136] The method illustrated in FIG. 14 may be an embodiment of step S1301 of FIG. 13.

[0137] The method illustrated in FIG. 14 can be performed, for example, by the host (100) illustrated in FIG. 1 or the host (100) illustrated in FIG. 2.

[0138] Referring to FIG. 14, in step S1401, the host (100) can generate a baseline based on the number of multiple feature tables and the number of all elements included in the multiple feature tables.

[0139] In step S1403, the host (100) can sort the multiple feature tables in ascending order based on the number of elements included in each of the multiple feature tables.

[0140] In step S1405, the host (100) may determine the feature tables located below the baseline among the plurality of feature tables as the first feature table group, and the feature tables located above the baseline among the plurality of feature tables as the second feature table group. Explanation of the symbols

[0141] 10: Computing Systems 100: Host 200: Memory System

Claims

Claim 1 A computing system comprising: a host that divides a plurality of feature tables, each containing a plurality of embedding vectors for a plurality of elements, into a first feature table group and a second feature table group, generates a first embedding table composed of the first feature table group, and requests a generation operation for a second embedding table composed of the second feature table group; and a memory system that generates the second embedding table in accordance with the request for the generation operation provided by the host, wherein the host divides the plurality of feature tables into the first feature table group and the second feature table group based on the number of elements included in each of the plurality of feature tables, and each of the plurality of feature tables includes embedding vectors for elements included in the corresponding feature that are quantified based on the similarity between elements included in the corresponding feature. Claim 2 delete Claim 3 A computing system according to claim 1, wherein the host generates a baseline based on the number of the plurality of feature tables and the number of all elements included in the plurality of feature tables, and divides the plurality of feature tables into the first feature table group and the second feature table group based on the baseline. Claim 4 In claim 3, the baseline is a computing system that is the result of an arithmetic mean using the number of the plurality of feature tables and the number of all elements. Claim 5 A computing system according to claim 3, wherein the host sorts the plurality of feature tables in ascending order based on the number of elements included in each of the plurality of feature tables, determines the feature tables among the plurality of feature tables located below the reference line as the first feature table group, and determines the feature tables among the plurality of feature tables located above the reference line as the second feature table group. Claim 6 A computing system according to claim 1, wherein the memory system receives information regarding the size of the second embedding table from the host in response to a request for the generation operation, and allocates a memory area to store the second embedding table based on the information regarding the size. Claim 7 In claim 6, the memory system is a computing system that, after allocating the memory area, provides the address of the memory area to the host. Claim 8 In claim 1, the host is a computing system that updates the first embedding table based on weights obtained through embedding learning and requests the memory system to update the second embedding table based on the weights. Claim 9 In claim 1, the host is a computing system that obtains an embedding vector of the target element by performing an embedding operation based on the first embedding table when a feature table containing the target element is included in the first feature table group. Claim 10 In claim 1, the host is a computing system that requests an embedding vector of the target element from the memory system when the feature table containing the target element is included in the second feature table group. Claim 11 In claim 10, the memory system is a computing system that, in response to a request for an embedding vector of the target element, performs an embedding operation based on the second embedding table to obtain an embedding vector of the target element and provides the obtained embedding vector to the host. Claim 12 A computing system according to claim 1, wherein the memory system comprises: a plurality of memory devices storing the second embedding table; a memory interface relaying packets between the host and the memory system; an embedding table management unit generating the second embedding table; and a computation unit performing an embedding operation based on the second embedding table. Claim 13 An embedding table partitioning unit that divides a plurality of feature tables, each containing a plurality of embedding vectors for a plurality of elements, into a first feature table group and a second feature table group; an embedding table management unit that generates a first embedding table composed of the first feature table group; and an embedding operation control unit that requests a memory system to generate a second embedding table composed of the second feature table group, wherein the embedding table partitioning unit divides the plurality of feature tables into the first feature table group and the second feature table group based on the number of elements included in each of the plurality of feature tables, and each of the plurality of feature tables includes embedding vectors for elements included in the corresponding feature that are quantified based on the similarity between the elements included in the corresponding feature. Claim 14 delete Claim 15 In claim 13, the embedding table partitioning unit generates a baseline based on the number of the plurality of feature tables and the number of all elements included in the plurality of feature tables, and divides the plurality of feature tables into the first feature table group and the second feature table group based on the baseline. Claim 16 In claim 15, the baseline is a host, which is the result of the arithmetic mean using the number of the plurality of feature tables and the number of all elements. Claim 17 In claim 15, the embedding table partitioning unit sorts the plurality of feature tables in ascending order based on the number of elements included in each of the plurality of feature tables, determines the feature tables among the plurality of feature tables located below the reference line as the first feature table group, and determines the feature tables among the plurality of feature tables located above the reference line as the second feature table group. Claim 18 In claim 13, the embedding table management unit updates the first embedding table based on weights obtained through embedding learning, and the embedding operation control unit requests the memory system to update the second embedding table based on the weights. Claim 19 In claim 13, the embedding operation control unit is a host that, when a feature table containing a target element is included in the first feature table group, performs an embedding operation based on the first embedding table to obtain an embedding vector of the target element. Claim 20 In claim 13, the embedding operation control unit is a host that requests an embedding vector of the target element from the memory system and receives the embedding vector of the target element from the memory system when the feature table containing the target element is included in the second feature table group. Claim 21 A computing system comprising: an embedding table partitioning unit that divides a plurality of feature tables, each containing a plurality of embedding vectors for a plurality of elements, into a plurality of feature table groups; and a plurality of computation processing units that generate an embedding table composed of one of the plurality of feature table groups, wherein the embedding table partitioning unit divides the plurality of feature tables into the plurality of feature table groups based on the number of elements included in each of the plurality of feature tables, and each of the plurality of feature tables includes embedding vectors for elements included in the corresponding features that are quantified based on the similarity between the elements included in the corresponding features. Claim 22 delete Claim 23 In claim 21, the plurality of computational processing devices are a computing system comprising a host and a plurality of memory systems.

Images