Near memory device, memory device, and electronic device

Abstract

A memory device, an electronic device, and a near memory device are provided. The memory device includes a near memory device and a far memory device, where the far memory device stores a compressed embedded vector, the near memory device includes a buffer that receives a request for pooling the embedded vector and outputs a hit signal or a missing signal in response to the request, the hit signal comprises an embedded vector corresponding to the request; an address calculation circuit that calculates a starting memory address and a memory size of the embedded vector in response to the missing signal; a decoding circuit that obtains a compressed embedded vector corresponding to the request from the remote memory device based on the starting memory address and the memory size, and outputs an embedded vector corresponding to the request by decoding the compressed embedded vector; and a pooling circuit that outputs a pooled embedded vector by performing a pooling operation on the embedded vector.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority from Korean Patent Application No. 10-2024-0018396, filed on February 6, 2024, and Korean Patent Application No. 10-2024-0052397, filed on April 18, 2024, in the Korean Intellectual Property Office, the disclosures of which are incorporated herein by reference in their entirety. Technical Field

[0003] The inventive concept relates to a memory device, and more particularly, to a memory device including a near memory device and a far memory device. Background Art

[0004] Electronic devices typically include a processor that performs operations and a memory device that stores the results of the operations. The memory device writes data in response to a write request and retrieves the stored data in response to a read request. Memory devices can be classified as volatile memory devices (such as dynamic random access memory (DRAM) and static RAM (SRAM)) that lose stored data when power is interrupted, or non-volatile memory devices (such as flash memory, phase change RAM (PRAM), magnetic RAM (MRAM), and resistive RAM (RRAM)) that retain data even when power is lost. Summary of the Invention

[0005] Embodiments of the inventive concept provide a method for quantizing and compressing embedded vector elements, thereby effectively reducing the data size of an embedding table.

[0006] According to an embodiment of the present invention, there is provided a memory device including a near memory device and a far memory device, wherein the far memory device stores a compressed embedded vector, the near memory device including: a buffer configured to receive a request for pooling an embedded vector and output a hit signal or a miss signal in response to the request, wherein the hit signal includes the embedded vector corresponding to the request; an address calculation circuit configured to calculate a starting memory address and a memory size of the embedded vector corresponding to the request in response to the miss signal; a decoding circuit configured to obtain a compressed embedded vector corresponding to the request from the far memory device based on the starting memory address and the memory size, and output the embedded vector corresponding to the request by decoding the compressed embedded vector; and a pooling circuit configured to output a pooled embedded vector by performing a pooling operation on the embedded vector, wherein the pooling circuit obtains the embedded vector from the hit signal or obtains the embedded vector from the decoding circuit, and the buffer stores the embedded vector having an access frequency higher than a preset value among all embedded vectors.

[0007] According to an embodiment of the present invention, there is provided an electronic device including a near memory device and a far memory device, the electronic device including a processor configured to control the near memory device and the far memory device, wherein the far memory device stores a compressed embedded vector, and the near memory device includes: a buffer configured to receive a request for pooling the embedded vector and output a hit signal or a miss signal in response to the request, wherein the hit signal includes the embedded vector corresponding to the request; an address calculation circuit configured to calculate a starting memory address and a memory size of the embedded vector corresponding to the request in response to the miss signal; a decoding circuit configured to obtain a compressed embedded vector corresponding to the request from the far memory device based on the starting memory address and the memory size, and output the embedded vector corresponding to the request by decoding the compressed embedded vector; and a pooling circuit configured to output a pooled embedded vector by performing a pooling operation on the embedded vector, wherein the pooling circuit obtains the embedded vector from the hit signal of the buffer or obtains the embedded vector from the decoding circuit, and the buffer stores the embedded vector having an access frequency higher than a preset value among all embedded vectors.

[0008] According to an embodiment of the present invention, there is provided a near memory device, comprising: a buffer configured to receive a request for pooling embedded vectors and output a hit signal or a miss signal in response to the request, wherein the hit signal includes the embedded vector corresponding to the request; an address calculation circuit configured to calculate a starting memory address and a memory size of the embedded vector corresponding to the request in response to the miss signal; a decoding circuit configured to obtain a compressed embedded vector corresponding to the request from a far memory device based on the starting memory address and the memory size of the embedded vector, and output the embedded vector corresponding to the request by decoding the compressed embedded vector; and a pooling circuit configured to output a pooled embedded vector by performing a pooling operation on the embedded vector, wherein the pooling circuit obtains the embedded vector from the hit signal of the buffer or obtains the embedded vector from the decoding circuit, and wherein the buffer stores embedded vectors having an access frequency higher than a preset value among the embedded vectors. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Embodiments of the present inventive concept will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0010] Figure 1 is a block diagram of an electronic device according to an embodiment;

[0011] Figure 2 is a block diagram illustrating a deep neural network (DNN) implemented by an electronic device according to an embodiment;

[0012] Figure 3 is a block diagram of a memory device according to an embodiment;

[0013] Figure 4 is a block diagram illustrating an operation of an electronic device in an offline section according to an embodiment;

[0014] Figure 5 is a block diagram illustrating a quantization operation and a compression operation according to an embodiment;

[0015] Figure 6 is an index table according to an embodiment;

[0016] Figure 7 is a diagram illustrating a mapping table according to an embodiment;

[0017] Figure 8 is a diagram illustrating a mapping table address table according to an embodiment;

[0018] Figure 9 is a block diagram illustrating the operation of an electronic device in an online portion according to an embodiment;

[0019] Figure 10 is a block diagram illustrating the operation of a buffer included in a near memory device in an online portion according to an embodiment;

[0020] Figure 11 is a block diagram illustrating the operation of an address calculation circuit included in a near memory device in an online portion according to an embodiment;

[0021] Figure 12 is a block diagram illustrating the operation of a decoding circuit included in a near memory device in an online portion according to an embodiment; and

[0022] Figure 13 is a block diagram illustrating an electronic device according to an embodiment of the inventive concept. DETAILED DESCRIPTION

[0023] Figure 1 is a block diagram of an electronic device according to an embodiment.

[0024] Reference Figure 1 , the electronic device 1 may include a processor 10 and a memory device 20. The memory device 20 may include a near memory device 100 and a far memory device 200. The electronic device 1 may be a computing system configured to process various information, such as a personal computer (PC), a server, a workstation, a tablet PC, a smart phone, a digital camera, a dashboard camera, etc.

[0025] The processor 10 may also be referred to as a system on a chip (SoC). The SoC may refer to a chip in which various circuits and modules performing different operations are implemented into a single integrated circuit.

[0026] For example, the processor 10 may include a central processing unit (CPU) and a cache memory. The CPU may control the overall operation of the electronic device 1. The cache memory may serve as a buffer memory for high-speed operation of the processor 10. According to some embodiments, the operating speed of the processor 10 may be faster than the data input / output speed of an off-chip memory (such as a near memory device 100 and a far memory device 200). In order to reduce the bottleneck caused by these speed differences, the cache memory may provide a high-speed buffer memory that supports the processor 10. In addition, according to some embodiments, the processor 10 may include an accelerator that assists its operation. For example, the accelerator may be a hardware device for performing graphics processing, machine learning, neural network calculations, etc.

[0027] The processor 10 may control the overall operation of the electronic device 1. The processor 10 may communicate with the near memory device 100 and the far memory device 200.

[0028] For example, the processor 10 may control the overall operation of the near memory device 100. The processor 10 may store data in the near memory device 100 or may read data stored in the near memory device 100. In addition, the processor 10 may control the overall operation of the far memory device 200. The processor 10 may store data in the far memory device 200 or may read data stored in the far memory device 200.

[0029] The near memory device 100 may store data or output stored data based on communication with the processor 10. For example, the near memory device 100 may be a volatile memory device such as a dynamic random access memory (DRAM) and may be packaged together with the processor 10. The near memory device 100 is located physically closer to the processor 10 than the far memory device 200, and the near memory device 100 provides superior latency and data input / output speed than the far memory device 200.

[0030] The far memory device 200 may store data or output stored data based on communication with the processor 10. For example, the far memory device 200 may be a volatile memory device such as a DRAM and may be housed in a package separate from the processor 10. The far memory device 200 is located farther away from the processor 10 than the near memory device 100, and the far memory device 200 may be superior in terms of additional connectivity and greater expansion capacity.

[0031] According to some embodiments, the near memory device 100 and the far memory device 200 may be manufactured using different processes. For example, the near memory device 100 may exhibit lower latency and faster data input / output speed than the far memory device 200, but the near memory device 100 may have a smaller storage capacity than the far memory device 200. The near memory device 100 may be manufactured using a logic process. According to another embodiment, the far memory device 200 may have a larger storage capacity than the near memory device 100, but may exhibit higher latency and slower data input / output speed than the near memory device 100. The far memory device 200 may be manufactured using a conventional process. However, the present inventive concept is not limited thereto, and each of the near memory device 100 and the far memory device 200 may be manufactured using a combination of a logic process and a conventional process.

[0032] Furthermore, according to some embodiments, the far memory device 200 may include or refer to a storage device. The storage device may store data or output stored data under the control of the processor 10. Compared to the cache memory, the near memory device 100, and the far memory device 200, the storage device may exhibit a relatively slow data input / output speed and a large storage capacity. For example, the storage device may be a non-volatile memory device such as a flash memory device.

[0033] The near memory device 100 may perform a pooling operation on the embedded vectors using near memory processing. In other words, the near memory device 100 may perform a pooling operation on the embedded vectors using near memory processing.

[0034] The following will refer to Figures 2 to 12 Give its detailed description.

[0035] By executing Figures 2 to 12 The quantization and compression method for embedding vector components described in can effectively reduce the data size of the embedding table. In addition, by performing Figures 2 to 12 The near-memory processing described in

[15] can effectively reduce the time and energy consumption for embedding table lookup and pooling.

[0036] Figure 2 2 is a block diagram illustrating a deep neural network (DNN) implemented by an electronic device according to an embodiment. Hereinafter, the DNN may be referred to as a neural network.

[0037] Reference above Figure 1 The described near-memory processing of the electronic device 1 can be applied to various neural networks.

[0038] According to some embodiments, the above reference Figure 1 The near memory processing of the electronic device 1 described can be applied to Figure 2Here, the neural network 2 may refer to a deep learning recommendation model (DLRM).

[0039] The neural network 2 may include a bottom multilayer perceptron (MLP) 51, an embedding layer 53, a pooling layer 55, an interaction layer 57, and a top MLP 59. The embedding layer 53 and the pooling layer 55 may be referred to above. Figure 1 The described near memory processing of the electronic device 1 is implemented.

[0040] The neural network 2 can be trained with data related to user preferences, making it well suited for recommending products customized for a user based on the preferences of multiple users.

[0041] Training the neural network 2 may include repeatedly performing multiple learning phases. Each training phase may include: a forward propagation operation for calculating the value of a loss function for a given set of weights; a calculation operation for determining at least one gradient of the loss function for the given weights; and a backward propagation operation for updating the weights based on the calculated gradient.

[0042] In the forward propagation operation during the inference phase of the neural network 2 and the training phase of the neural network 2, the neural network 2 may receive a first input 1 st INPUT and second input 2 nd INPUT as input value.

[0043] In detail, the first input 1 st INPUT can be processed by the bottom MLP 51, and the second input 2 nd The INPUT may be processed by the embedding layer 53 and the pooling layer 55 .

[0044] First input 1 st INPUT can be an input including continuous features and dense features. Continuous features can refer to features expressed by continuous values, such as price, age, annual salary, etc. Dense features refer to the case where the number of elements with value 0 in the input represented as a matrix or vector (i.e., input matrix or input vector) is less than a preset number. In other words, the first input 1 st INPUT is represented as an input matrix (or input vector) containing continuous values, wherein the number of elements having a value of 0 among the elements of the input matrix (or input vector) may be less than a preset number.

[0045] In addition, the second input 2 ndINPUT can be an input containing categorical features and sparse features. Categorical features can refer to features expressed by discrete values, such as gender and age. Sparse features refer to the case where the number of elements with value 0 in the input represented as a matrix or vector (i.e., input matrix or input vector) is greater than or equal to a preset number. In other words, the second input 2 nd INPUT is represented as an input matrix (or input vector) containing discrete values, wherein the number of elements having a value of 0 among the elements of the input matrix (or input vector) may be greater than or equal to a preset number.

[0046] In other words, the neural network 2 may receive both continuous features and dense features as well as categorical features and sparse features as input values, and may process both continuous features and dense features as well as categorical features and sparse features.

[0047] First input 1 st INPUT can be processed by the bottom MLP 51. The bottom MLP 51 can receive the first input 1 st INPUT as input value, and will pass the first input 1 st INPUT The first input processed by performing the forward propagation operation PROCESSED 1 st INPUT is provided to the interaction layer 57.

[0048] Second input 2 nd INPUT can be processed by the embedding layer 53 and the pooling layer 55. Second input 2 nd The INPUT may include at least one table index and at least one row index.

[0049] The embedding layer 53 may include multiple embedding tables. Figure 2 When the neural network 2 shown is a deep learning recommendation model (DLRM), each embedding table can represent user-related and / or item-related information. The purpose of generating a pooled embedding vector (POOLED EMBEDDING VECTOR) by accessing multiple embedding tables is to extract information related to interactions between specific users, user groups, specific items, or groups of items.

[0050] In addition, an embedding table may include multiple embedding vectors. Each element included in each of the embedding vectors may be in 32-bit single-precision floating point format. However, the present invention is not limited thereto, and each element included in each of the embedding vectors may also be expressed in other forms.

[0051] The embedding layer 53 may identify a corresponding embedding table from a plurality of embedding tables based on the table index. Furthermore, the embedding layer 53 may identify a corresponding row from a plurality of rows included in the corresponding embedding table based on the row index. In other words, the embedding layer 53 may provide an embedding vector corresponding to a specific row of the embedding table identified by the table index and the row index to the pooling layer 55.

[0052] In other words, the embedding layer 53 can be based on the second input 2 nd The table index and row index of INPUT perform an embedding table lookup operation, thereby providing at least one embedding vector to the pooling layer 55.

[0053] The pooling layer 55 may receive at least one embedding vector as an input value and perform a pooling operation on the at least one embedding vector, thereby providing a pooled embedding vector POOLED EMBEDDING VECTOR to the interaction layer 57. However, the present invention is not limited thereto, and the pooling operation may be a concatenation operation or a multiplication operation for an element.

[0054] The processing operations of the embedding layer 53 and the pooling layer 55 can be implemented as described above with reference to Figure 1 The near memory processing within the electronic device 1 is described.

[0055] The interaction layer 57 may receive the processed first input PROCESSED 1 st INPUT and pooled embedding vector POOLEDEMBEDDING VECTOR as input values. Then, the interaction layer 57 can process the processed first input PROCESSED 1 based on the feature interaction function. st INPUT is combined with the pooling embedding vector POOLED EMBEDDING VECTOR to provide an intermediate output value OUTPUT_IN to the top MLP 59.

[0056] The top MLP 59 may receive the intermediate output value OUTPUT_IN as an input value and output a final output value OUTPUT by performing a forward propagation operation on the intermediate output value OUTPUT_IN.

[0057] For example, when Figure 2 When the neural network 2 shown involves DLRM, the final output value OUTPUT may represent a predicted click-through rate (CTR).

[0058] The processing operations of the bottom MLP 51, the interaction layer 57 and the top MLP 59 can be referred to above. Figure 1 The processor 10 is implemented. Here, by referring to the above Figure 1The bottom MLP 51 and / or the top MLP 59 of the depicted processor 10 implementation may perform weight applications and multiply and accumulate (MAC) operations on input values.

[0059] According to the present invention, the processing operations of the embedding layer 53 and the pooling layer 55 can be implemented as described above with reference to Figure 1 The near memory processing within the electronic device 1 is described.

[0060] Figure 3 is a block diagram of a memory device according to an embodiment.

[0061] Reference Figure 3 , the near memory device 100 may include a buffer 110 , an address calculation circuit 120 , a decoding circuit 130 , and a pooling circuit 140 .

[0062] The buffer 110 may receive a request REQUEST from the processor 10. In addition, the buffer 110 may output a hit signal HIT or a miss signal MISS in response to the request REQUEST. The hit signal HIT may include an embedded vector corresponding to the request REQUEST.

[0063] The buffer 110 may store embedded vectors having an access frequency higher than a preset value among embedded vectors included in the plurality of embedding tables. The stored embedded vectors may be referred to as frequently accessed vectors (FAV) 31.

[0064] The buffer 110 may output a hit signal HIT when the embedded vector corresponding to the request REQUEST matches one of the stored embedded vectors, and may output a miss signal MISS when the embedded vector corresponding to the request REQUEST does not match one of the stored embedded vectors.

[0065] According to some embodiments, the buffer 110 may be implemented using a memory. For example, the buffer 110 may be implemented using a volatile memory such as DRAM, mobile DRAM, static RAM (SRAM), phase change RAM (PRAM), ferroelectric RAM (FRAM), resistive RAM (RRAM), and / or magnetic RAM (MRAM).

[0066] The address calculation circuit 120 may calculate a start memory address and a memory size of an embedded vector corresponding to the request REQUEST in response to the miss signal MISS from the buffer 110 .

[0067] The decoding circuit 130 may obtain the compressed embedded vector corresponding to the request REQUEST from the far memory device 200 based on the calculated starting memory address and the calculated memory size of the embedded vector corresponding to the request REQUEST.

[0068] In addition, the decoding circuit 130 may output an embedded vector corresponding to the request REQUEST by decoding the compressed embedded vector.

[0069] The decoding circuit 130 may store an index table 33. In addition, the address calculation circuit 120 may store a mapping table (MT) address table 35 and a CET address table 37. A detailed description thereof will be given later with reference to another drawing.

[0070] The pooling circuit 140 may output a pooled embedding vector by performing a pooling operation on the embedding vector.

[0071] The pooling circuit 140 may obtain the embedded vector from the hit signal HIT of the buffer 110 , or obtain the embedded vector from the decoding circuit in response to the miss signal MISS of the buffer 110 .

[0072] Hereinafter, the near memory processing of the electronic device 1 will be described by dividing the near memory processing into an offline part and an online part. Figures 4 to 8 Describes the offline part and will refer to Figures 9 to 12 Describes the online part.

[0073] Figure 4 is a block diagram illustrating an operation of an electronic device in an offline section according to an embodiment. Figure 5 is a block diagram illustrating a quantization operation and a compression operation according to an embodiment. Figure 6 is an index table according to an embodiment. Figure 7 is a diagram illustrating a mapping table according to an embodiment. Figure 8 is a diagram illustrating an MT address table according to an embodiment.

[0074] Figure 4 is a block diagram for describing a near memory process in an offline section of the electronic device 1 according to the embodiment.

[0075] The offline part may refer to a part in which data or tasks are prepared or processed in advance. In other words, the offline part refers to a stage in which data or tasks are prepared or processed in advance. For example, referring to Figure 2 The process of training the neural network 2 described above can be included in an offline portion. In other words, in the offline portion, the electronic device 1 can train the neural network 2 based on the training data. The trained neural network 2 can be used in a subsequent online portion.

[0076] In addition, when the training of the neural network 2 is completed, the electronic device 1 may generate an embedding vector of the trained neural network 2. In addition, the electronic device 1 may collect the embedding vectors and generate a plurality of embedding tables ET.

[0077] According to the inventive concept, the electronic device 1 may also perform a process of quantizing elements of each embedding vector of the neural network 2 trained in the offline portion and compressing the quantized elements.

[0078] Reference Figure 4 , the electronic device 1 may generate a plurality of quantization embedding tables ET_Q by quantizing elements of the embedding vectors included in the plurality of embedding tables ET.

[0079] The electronic device 1 may generate a plurality of compressed embedding tables 41 by compressing a plurality of quantization embedding tables ET_Q. Here, the electronic device 1 may store the plurality of compressed embedding tables 41 in a reference Figure 1 In the far memory device 200 described.

[0080] The electronic device 1 may generate metadata by compressing a plurality of quantization embedding tables ET_Q. The metadata may include reference Figure 5 and Figure 6 Index table 33 of description, reference Figure 7 The mapping table 43 described, and the reference Figure 8 Described MT address table 35 and CET address table 37.

[0081] Will refer to Figure 5 A quantization operation and a compression operation according to an embodiment are described.

[0082] Reference Figure 5 , shows one embedded vector element among the elements in each embedded vector included in each embedding table ET in the plurality of embedding tables ET. In other words, shows one element from each embedded vector in the plurality of embedding tables ET.

[0083] Each element included in each of the embedded vectors may be in a 32-bit single-precision floating point format. In other words, one embedded vector element may be a value in a 32-bit floating point (FP32) format.

[0084] Reference Figure 5 , one embedded vector element may include a sign bit S allocated with 1 bit, an exponent bit E allocated with 8 bits, and a mantissa bit M allocated with 23 bits.

[0085] The electronic device 1 may perform the quantization operation by retaining a preset number of most significant bits of the mantissa bits M of each element in each embedding vector from each embedding table ET of the plurality of embedding tables ET and removing the remaining mantissa bits M.

[0086] Reference Figure 5, the electronic device 1 may quantize the one embedded vector element by retaining the most significant bit MSB, the second most significant bit SMBS, and the third most significant bit TMSB from the mantissa bits M of the one embedded vector element and removing the remaining bits of the mantissa bits M. For example, the electronic device 1 may quantize the embedded vector element by retaining the most significant bit MSB, the second most significant bit SMBS, and the third most significant bit TMSB from the mantissa bits M of the embedded vector element and removing the remaining mantissa bits M.

[0087] In other words, the electronic device 1 may quantize one embedded vector element by leaving only the three most significant bits of the mantissa bits M and removing the remaining 20 bits of the mantissa bits M, thereby generating a quantized embedded vector element.

[0088] According to the present invention, by leaving only the three most significant bits of the mantissa bits M and removing the remaining 20 bits of the mantissa bits M, the quantized embedded vector elements can closely approximate the original values, thereby minimizing their impact on the neural network 2.

[0089] The electronic device 1 may generate a plurality of quantization embedding tables ET_Q by retaining a preset number of most significant bits from among the mantissa bits M of each element of each embedding vector included in each embedding table ET of the plurality of embedding tables ET and removing the remaining bits of the mantissa bits M. In other words, the electronic device 1 may generate a plurality of quantization embedding tables ET_Q by retaining a preset number of most significant bits from among the mantissa bits M of each element in the embedding vector within the embedding table ET and removing the remaining mantissa bits M.

[0090] In addition, based on the bit value of the exponent bit E of each quantized embedded vector element of each embedded vector included in each quantized embedded table ET_Q in multiple quantized embedded tables ET_Q, the electronic device 1 can map each exponent bit E in the exponent bit E to an exponent bit E having a preset number of bits, or add a preset number of bits with a specific bit value between the sign bit S and the exponent bit E.

[0091] Reference Figure 5 , based on the bit value of the exponent bit E of a quantized embedded vector element, the electronic device 1 may map each exponent bit E to an exponent bit E having two bits (Compressing_case1), or add two bits having a specific bit value between the sign bit S and the exponent bit E (Compressing_case2).

[0092] The bit value of the exponent bit E with two bits to be mapped can be 00 (2) 、01 (2) and 10 (2) The value of the two bits added between the sign bit S and the exponent bit E can be 11 (2) .

[0093] In other words, the electronic device 1 may generate one compressed embedded vector element by mapping each exponent bit E to an exponent bit E having two bits (Compressing_case1) or adding two bits having a specific bit value between the sign bit S and the exponent bit E (Compressing_case2).

[0094] Reference Figure 5 , when the exponent bit E of one quantized embedded vector element is mapped to the exponent bit E having two bits (Compressing_case1), the size of one compressed embedded vector element may be 6 bits in total.

[0095] Reference Figure 5 , when two bits with specific bit values are added between the sign bit S and the exponent bit E of one quantized embedded vector element (Compresing_case2), the size of one compressed embedded vector element may be a total of 14 bits.

[0096] According to the concept of the present invention, lossless compression of quantized embedded vector elements can be achieved by mapping each exponent bit E to an exponent bit E with two bits (Compressing_case1) or adding two bits with specific bit values between the sign bit S and the exponent bit E (Compressing_case2).

[0097] This is because more than 80% of the exponent bits E of the elements in the embedded vector of the trained neural network 2 each have one of three values: a first value, a second value, and a third value. The first value can be referred to as top-1, the second value can be referred to as top-2, and the third value can be referred to as top-3. In other words, the electronic device 1 can compress the embedded vector by mapping the three most frequently occurring exponent bits E to the forms of (00)2, (01)2, and (10)2, respectively.

[0098] Furthermore, by mapping the exponent bit E to the exponent bit E having two bits (Compressing_case1), the electronic device 1 can generate an exponent table 33 including corresponding first bit values, corresponding second bit values, and corresponding third bit values respectively corresponding to the plurality of embedding tables ET.

[0099] Return to reference Figure 4 , the metadata may include an index table 33. The index table 33 may be an index table corresponding to the above references Figure 6 Tables of top-1, top-2, and top-3 bit values of multiple embedding tables ET are described.

[0100] In other words, the exponent table 33 may include metadata for a 2-bit mapping (Compressing_case1) of the exponent bits E.

[0101] Reference Figure 4 In the offline portion, the electronic device 1 may store the generated index table 33 in the decoding circuit 130 of the near memory device 100 .

[0102] Reference Figure 6 , it can be seen that the index table of N embedding tables is shown. Here, N is a positive integer.

[0103] Reference Figure 6 , in the 0th embedding table (0), the exponent bit E is 01100011 (2) Can be mapped to 00 (2) , exponent bit E01100100 (2) Can be mapped to 01 (2) , and the exponent is E01100101 (2) Can be mapped to 10 (2) In the first embedding table (1), the exponent is E01100100 (2) Can be mapped to 00 (2) , exponent bit E 01110000 (2) Can be mapped to 01 (2) , exponent bit E01110011 (2) Can be mapped to 10 (2) In the N-1th embedded table (N-1), the exponent bit E is 01111000 (2) Can be mapped to 00 (2) , exponent bit E 01111100 (2) Can be mapped to 01 (2) , and the exponent bit is E 01100011 (2) Can be mapped to 10 (2) .

[0104] Return to reference Figure 4 , the metadata may include a mapping table 43. The mapping table 43 may represent metadata for calculating a starting memory address for each embedded vector.

[0105] According to an embodiment, the value of each row of the mapping table of one embedded table may represent the number of elements compressed to 14 bits among the elements from row 0 (ie, initial row) to the row immediately before the corresponding row of the mapping table ( Figure 5 Compressing_case2).

[0106] For example, the value of the 0th row (i.e., the initial row) of the mapping table of the embedded table can be 0, the value of the first row can represent the number of elements compressed to 14 bits from the elements corresponding to the 0th row, and the value of the second row can represent the number of elements compressed to 14 bits from the elements corresponding to the 0th row and the first row.

[0107] Reference Figure 7 , showing a mapping table of N embedded tables. Here, N is a positive integer.

[0108] Reference Figure 7 , in the 0th mapping table of the 0th embedding table, the value of the 0th row may be 0. In the 0th mapping table of the 0th embedding table, the value 1 of the first row may indicate that the number of elements compressed to 14 bits among the elements of the embedded vector corresponding to the 0th row is 1, the value 5 of the second row may indicate that the number of elements compressed to 14 bits among the elements of the embedded vectors corresponding to the 0th to the first row (that is, the embedded vector corresponding to the 0th row and the embedded vector corresponding to the first row) is 5, and the value 8 of the third row may indicate that the number of elements compressed to 14 bits among the elements of the embedded vectors corresponding to the 0th to the second row (that is, the embedded vector corresponding to the 0th row, the embedded vector corresponding to the first row, and the embedded vector corresponding to the second row) is 8.

[0109] Furthermore, in the offline portion, the electronic device 1 may store the mapping table 43 in the far memory device 200. Here, since the size of the mapping table 43 is large, the electronic device 1 may store the mapping table in the far memory device 200.

[0110] Furthermore, since the mapping table 43 is stored in the far memory device 200, the near memory device 100 may store the MT address to access the mapping table 43 stored in the far memory device 200. The MT address may refer to the physical memory address of the first row of the corresponding mapping table. In other words, the MT address may be the physical memory address of the far memory device 200 for the first row of the corresponding mapping table.

[0111] Return to reference Figure 4 , the metadata may include an MT address table 35. The MT address table 35 may be a table representing all MT addresses.

[0112] In other words, the MT address table 35 may contain metadata associated with the physical memory address of the first row in all mapping tables.

[0113] Reference Figure 4 , the electronic device 1 may generate an MT address table 35 including the above-mentioned MT addresses, and the generated MT address table 35 may be stored in the address calculation circuit 120 of the near memory device 100 .

[0114] In other words, in the offline portion, the electronic device 1 may store the MT address table 35 in the address calculation circuit 120 of the near memory device 100 .

[0115] Reference Figure 8 , showing Figure 7 MT address table of the mapping table.

[0116] Reference Figure 7 and Figure 8 , the value 0x00000040 corresponding to the 0th row of the MT address table may represent the physical memory of the 0th row (i.e., the initial row) of the 0th mapping table. The value 0x00000140 corresponding to the first row of the MT address table may represent the physical memory of the 0th row (i.e., the initial row) of the first mapping table. The value 0x00001040 corresponding to the (N-1)th row of the MT address table may represent the physical memory of the 0th row (i.e., the initial row) of the (N-1)th mapping table.

[0117] The near memory device 100 may store a compressed embedding table (CET) address to access the compressed embedding table 41 stored in the far memory device 200. The CET address may refer to the physical memory address of the first row of the corresponding embedding table. In other words, the CET address may be the physical memory address of the far memory device 200 for the first row of the corresponding embedding table.

[0118] Return to reference Figure 4 , the metadata may include a CET address table 37. Here, the CET address table 37 may be a table representing all CET addresses.

[0119] In other words, the CET address table 37 may include metadata regarding the physical memory addresses of the first rows of all embedded tables.

[0120] Reference Figure 4 , the electronic device 1 may generate a CET address table 37 including the above-mentioned CET addresses, and the generated CET address table 37 may be stored in the address calculation circuit 120 of the near memory device 100 .

[0121] In other words, in the offline portion, the electronic device 1 may store the CET address table 37 in the address calculation circuit 120 of the near memory device 100 .

[0122] The MT address table 35 and the CET address table 37 may be used later in the inline portion by the address calculation circuit 120 of the near memory device 100 .

[0123] return Figure 4, the electronic device 1 may extract an embedded vector whose access frequency is higher than a preset value from among the embedded vectors included in each of the plurality of embedding tables ET, and store the extracted embedded vector in the buffer 110 of the near memory device 100. Here, the extracted embedded vector may be referred to as a frequently accessed vector (FAV).

[0124] The above access refers to the reference Figure 2 The access frequency refers to the frequency of accesses to an embedding vector in the table relative to the total number of accesses to the multiple embedding vectors in the corresponding embedding table. According to some embodiments, the access frequency may be a statistical value based on the actual data set used to train the neural network 2.

[0125] The electronic device 1 may store the FAV 31 in the buffer 110 of the near memory device 100 by extracting the FAV 31 from the plurality of embedding tables ET based on access frequencies of the respective embedded vectors.

[0126] According to some embodiments, the access frequency may be defined as Equation 1 below.

[0127] [Equation 1]

[0128]

[0129] Here, Cov i It can represent the access frequency of the i-th embedded vector, AC i can represent the number of visits to the i-th embedded vector, and AC total It can represent the total number of accesses to all embedding vectors included in the embedding table including the i-th embedding vector. i Different from the hit rate, because the access frequency Cov i is a parameter for a specific embedding vector, and hit rate is a hit signal output frequency of the buffer 110 , eg, a parameter for the buffer 110 .

[0130] In other words, the electronic device 1 can extract the access frequency Cov i The embedded vector is higher than a preset value, and the extracted embedded vector is stored in the buffer 110 of the near memory device 100.

[0131] According to some embodiments, the electronic device 1 may embed the vectors having a high access frequency Cov among all embedding vectors included in one embedding table. i The 16 embedding vectors of the present invention are stored in the buffer 110 of the near memory device 100. In addition, according to some embodiments, the electronic device 1 may store the embedding vectors having a high access frequency Cov in all the embedding vectors included in one embedding table. iThe 64 embedded vectors are stored in the buffer 110 of the near memory device 100 .

[0132] Figure 9 is a block diagram illustrating an operation of an electronic device in an online section according to an embodiment. Figure 10 is a block diagram illustrating an operation of a buffer included in a near memory device in an online section according to an embodiment. Figure 11 is a block diagram illustrating the operation of an address calculation circuit included in a near memory device in an online section according to an embodiment. Figure 12 is a block diagram illustrating the operation of a decoding circuit included in a near memory device in an online portion according to an embodiment.

[0133] Here, the online part may refer to the part where data is processed in real time or tasks are executed in response to real-time situations. In other words, the online part refers to the stage where data is processed in real time or tasks are executed in response to real-time situations. For example, using reference Figure 2 The described process of neural network 2 making predictions about actual data may be included in the online portion. In other words, the online portion may include the portion where neural network 2 performs inference operations. During the online portion, neural network 2 may receive data in real time and perform inference operations on the data.

[0134] Reference Figure 9 , the processor 10 may output a request REQUEST for pooling of embedding vectors to the near memory device 100. Here, the request REQUEST may include an embedding table index including a corresponding embedding vector and an index of the corresponding embedding vector in the embedding table.

[0135] Reference Figure 9 , the buffer 110 may receive a request REQUEST from the processor 10. In addition, the buffer 110 may output a hit signal HIT or a miss signal MISS in response to the request REQUEST. The hit signal HIT may include an embedded vector corresponding to the request REQUEST.

[0136] Reference Figure 4 and Figure 9 , in the offline part, the FAV may be stored in the buffer 110. Also, in the online part, when the embedded vector corresponding to the request REQUEST matches one of the stored FAVs 31, the buffer 110 outputs a hit signal HIT, and when the embedded vector corresponding to the request REQUEST matches none of the stored FAVs 31, the buffer 110 outputs a miss signal MISS.

[0137] Figure 10 is a block diagram illustrating the buffer 110 according to an embodiment.

[0138] Reference Figure 10 , the buffer 110 may store the FAV 31 in a tag array format.

[0139] In the tag array format, each data block can be identified by a tag that includes an address. In other words, the tag is part of the memory address and can indicate the address of the data stored at a specific location in the memory.

[0140] Reference Figure 10 , the buffer 110 may access the tag array 50 to determine whether the corresponding embedding vector is stored in the FAV buffer.

[0141] The request REQUEST may include an embedding table index, the embedding table index including a corresponding embedding vector and an index of the corresponding embedding vector in the embedding table. In addition, the request REQUEST may be represented as a vector in which the value of the embedding table index and the value of the index of the embedding vector are concatenated. In addition, Figure 10 The tag array 50 shown may be a set of vectors in which, for all FAVs, the index value of the embedding table of the corresponding embedding vector and the index value of the corresponding embedded vector in the corresponding embedding table are concatenated. In other words, each tag may include a bit in which the embedding table number and the embedded vector row index in the corresponding embedding table are concatenated.

[0142] In other words, the buffer 110 may output a hit signal HIT or a miss signal MISS by comparing the value of the embedding table index included in the request REQUEST and the tag array 50 with the value of the index of the corresponding embedded vector.

[0143] When the value of the embedding table index included in the request REQUEST and the value of the index of the corresponding embedding vector in the embedding table are included in the tag array 50 (ie, when there is a match), the buffer 110 may output a hit signal HIT.

[0144] When the value of the embedding table index included in the request REQUEST and the value of the index of the corresponding embedding vector in the embedding table are not included in the tag array 50 (ie, there is a mismatch), the buffer 110 may output a missing signal MISS.

[0145] Return to reference Figure 9 In response to the hit signal HIT of the buffer 110 , the pooling circuit 140 may perform a pooling operation on the corresponding embedded vector.

[0146] Here, the pooling circuit 140 may receive the hit signal HIT and the signal EMBEDDING VECTOR including the embedding vector corresponding to the request REQUEST. According to an embodiment, the hit signal HIT may include the embedding vector corresponding to the request REQUEST.

[0147] In other words, the pooling circuit 140 may receive the hit signal HIT as an input value and output a pooled embedding vector POOLED EMBEDDING VECTOR by performing a pooling operation on the corresponding embedding vector.

[0148] Reference Figure 9 , the pooling circuit 140 can provide the pooled embedding vector POOLED EMBEDDINGVECTOR to the processor 10.

[0149] Reference Figure 9 In response to the miss signal MISS of the buffer 110, the address calculation circuit 120 may output a signal STARTING MEMORY ADDRESS AND MEMORY SIZE OF EMBEDDING VECTOR, which indicates a corresponding starting memory address and a corresponding memory size of the far memory device 200 in which the corresponding embedded vector is stored. In order for the decoding circuit 130 to access the corresponding compressed embedding table from the compressed embedding table 41 stored in the far memory device 200, the address calculation circuit 120 may provide the signal STARTING MEMORY ADDRESS AND MEMORY SIZE OF EMBEDDING VECTOR to the decoding circuit 130.

[0150] In other words, the address calculation circuit 120 may receive the missing signal MISS as an input value and output a signal STARTING MEMORY ADDRESS AND MEMORY SIZE OF EMBEDDING VECTOR by calculating a corresponding starting memory address and a corresponding memory size of the far memory device 200 in which the corresponding embedded vector is stored.

[0151] To obtain values required to calculate the starting memory address and the corresponding memory size of the far memory device 200 in which the corresponding embedded vector is stored, the address calculation circuit 120 may access the mapping table 43 stored in the far memory device 200 .

[0152] Figure 11 is a block diagram of the address calculation circuit 120 according to the embodiment.

[0153] Reference Figure 11 The address calculation circuit 120 may include an address decoding circuit 121 , a first register 123 , and a second register 125 .

[0154] In order to access the corresponding compressed embedding table from the compressed embedding table 41 stored in the far memory device 200, the starting memory address of the corresponding embedded vector and the memory size of the corresponding embedded vector may be required. Here, the starting memory address may be a physical memory address.

[0155] The first register 123 can store a reference Figure 4 and Figure 8 The first register 123 may provide the MT address included in the MT address table 35 to the address decoding circuit 121 .

[0156] The second register 125 can store a reference Figure 4 The second register 125 may be a CET address register and may provide the CET address included in the CET address table 37 to the address decoding circuit 121.

[0157] In response to the missing signal MISS, the address decoding circuit 121 may calculate a starting memory address and a memory size of the corresponding compressed embedded vector, thereby outputting a signal STARTING MEMORY ADDRESS AND MEMORY SIZE OF EMBEDDING VECTOR.

[0158] Furthermore, in response to the miss signal MISS, the address decoding circuit 121 may access the mapping table 43 stored in the far memory device 200 based on the MT address register corresponding to the corresponding embedded vector.

[0159] In other words, the address decoding circuit 121 may receive the missing signal MISS as an input value. In addition, the address decoding circuit 121 may use the MT address register corresponding to the embedded vector to access the mapping table 43 stored in the far memory device 200. This enables the address decoding circuit 121 to obtain the values required to calculate the starting memory address and memory size of the corresponding embedded vector from the mapping table 43.

[0160] According to an embodiment, the address decoding circuit 121 may calculate the starting memory address of the corresponding embedded vector based on the following Equation 2. Here, it is assumed that the corresponding embedded vector has an index N in the corresponding embedding table (ie, the corresponding embedded vector is located in the Nth row in the corresponding embedding table).

[0161] [Equation 2]

[0162] Addr S =Addr CET +6×d+8×MT[N-1]

[0163] Here, Addr SIt can represent the starting memory address of the corresponding embedded vector, Addr CET may represent a start memory address of a corresponding compressed embedding table, d may represent the total number of elements included in one embedding vector, and MT[N-1] may represent a value of an N-1th row of a mapping table corresponding to the corresponding embedding table.

[0164] In other words, the address decoding circuit 121 can be based on the starting memory address Addr of the corresponding compression embedding table. CET The starting memory address of the corresponding compressed embedded vector is calculated by using the value MT[N-1] of the row immediately before the corresponding row of the mapping table corresponding to the corresponding embedding table. Here, the value MT[N-1] of the row immediately before the corresponding row of the mapping table corresponding to the embedding table may correspond to the cumulative number of embedded vectors compressed to a size of 14 bits in the mapping table to the corresponding row (row N).

[0165] Furthermore, the address decoding circuit 121 may calculate a memory size of a corresponding embedded vector based on Equation 3 and Equation 4 below.

[0166] [Equation 3]

[0167] Addr N =Addr CET +6×d+8×MT[N]

[0168] [Equation 4]

[0169] Len V =Addr N -Addr S

[0170] Here, Addr N The starting memory address of the embedded vector located in the next row of the corresponding embedded vector (ie, the embedded vector located in the N+1th row of the corresponding embedding table) may be represented by Addr CET Len may represent the starting memory address of the corresponding compressed embedding table, d may represent the total number of elements included in one embedding vector, and MT[N] may represent the value of the Nth row of the mapping table corresponding to the corresponding embedding table. V The memory size of the corresponding embedded vector (ie, the length of the data) can be represented, and Addr S Indicates the starting memory address of the corresponding embedded vector.

[0171] In other words, the address decoding circuit 121 may be based on the starting memory address Addr of the corresponding embedded vector. S and the starting memory address Addr of the next embedded vector corresponding to the embedded vector N To calculate the memory size of the corresponding embedding vector.

[0172] Return to reference Figure 9 In response to the signal STARTING MEMORY ADDRESS AND MEMORY SIZE OF EMBEDDING VECTOR from the address calculation circuit 120, the decoding circuit 130 may access the compressed embedding table 41 stored in the far memory device 200 to obtain the corresponding compressed embedding vector.

[0173] In other words, the decoding circuit 130 may receive the signal STARTING MEMORY ADDRESS AND MEMORY SIZEOF EMBEDDING VECTOR as an input value and access the compressed embedding table 41 stored in the far memory device 200, thereby obtaining a corresponding compressed embedding vector included in the corresponding compressed embedding table.

[0174] Thereafter, the decoding circuit 130 may decode the corresponding compressed embedding vector to output a signal EMBEDDINGVECTOR including the embedding vector corresponding to the request REQUEST. The signal EMBEDDINGVECTOR is provided to the pooling circuit 140.

[0175] Figure 12 is a block diagram illustrating the operation of the decoding circuit 130 according to an embodiment.

[0176] Reference Figure 12 , shows an example of an index table of a compressed embedded vector and a 0th compressed embedded vector element CEV0, a first compressed embedded vector element CEV1, and a second compressed embedded vector element CEV2.

[0177] Here, the decoding circuit 130 may sequentially decode the elements of the embedded vector in synchronization with the clock signal. According to some embodiments, one compressed embedded vector element may be decoded per clock cycle.

[0178] The decoding circuit 130 may replace the bit values of the exponent bits E of the compressed embedded vector elements with the bit values included in the exponent table based on the bit values of the MSB and SMBS of the exponent bits E of the compressed embedded vector elements, or remove the MSB and SMBS from the exponent bits E of the compressed embedded vector elements.

[0179] Reference Figure 12, among the exponent bits E of the 0th compressed embedded vector element CEV0, the MSB is 0 and the SMBS is 0, and therefore, the bit value of the exponent bits E of the 0th compressed embedded vector element CEV0 (i.e., 00) may be replaced by 01100011 (see CYCLE1). Among the exponent bits E of the first compressed embedded vector element CEV1, the MSB is 1 and the SMBS is 1, and therefore, the bit value of the exponent bits E of the first compressed embedded vector element CEV1 (i.e., 11) may be removed (see CYCLE 2). Among the exponent bits E of the second compressed embedded vector element CEV2, the MSB is 1 and the SMBS is 0, and therefore, the bit value of the exponent bits E of the second compressed embedded vector element CEV2 (i.e., 10) may be replaced by 01100101 (see CYCLE 3).

[0180] Return to reference Figure 9 The decoding circuit 130 may decode the corresponding compressed embedded vector and provide the signal EMBEDDING VECTOR including the embedded vector corresponding to the request REQUEST to the pooling circuit 140.

[0181] In response to the signal EMBEEDDING VECTOR of the decoding circuit 130 , the pooling circuit 140 may perform a pooling operation on the corresponding embedding vector.

[0182] In other words, the pooling circuit 140 may receive the signal EMBEDDING VECTOR as an input value and output a pooled embedding vector POOLED EMBEDDING VECTOR by performing a pooling operation on the corresponding embedding vector. If the hit signal HIT is received, the pooling circuit 140 may provide the pooled embedding vector POOLED EMBEDDING VECTOR to the processor 10.

[0183] As described above, by performing the quantization and compression methods on the embedding vector elements, the data size of the embedding table can be effectively reduced.

[0184] Furthermore, by performing near-memory processing based on the description given above, both the time and energy consumption for embedding table lookup and pooling can be significantly reduced.

[0185] Figure 13 is a block diagram illustrating an electronic device according to an embodiment of the inventive concept.

[0186] Reference Figure 13 , Figure 13 The system 2000 may be a mobile system such as a mobile phone, a smart phone, a tablet personal computer (PC), a wearable device, a healthcare device, or an Internet of Things (IoT) device. However, Figure 13The system 2000 is not necessarily limited to a mobile system and may include a PC, a laptop computer, a server, a media player, or an automotive device such as a navigation device. The system 2000 may be referred to as an electronic device. Here, the system 2000 may correspond to Figure 1 electronic device 1.

[0187] Reference Figure 13 The system 2000 may include a main processor 2100, memories 2200a and 2200b, and storage devices 2300a and 2300b, and may additionally include at least one of an image capture device 2410, a user input device 2420, a sensor 2430, a communication device 2440, a display 2450, a speaker 2460, a power supply device 2470, and a connection interface 2480. The main processor 2100 may correspond to Figure 1 processor 10.

[0188] The main processor 2100 may control the overall operation of the system 2000, and more specifically, control operations of other components constituting the system 2000. The main processor 2100 may be implemented by a general-purpose processor, a dedicated processor, or an application processor.

[0189] The main processor 2100 may include one or more CPU cores 2110, and may also include a controller 2120 for controlling memories 2200a and 2200b and / or storage devices 2300a and 2300b. According to an embodiment, the main processor 2100 may also include an accelerator 2130, which is a dedicated circuit for high-speed data operations (such as artificial intelligence (AI) data operations). The accelerator 2130 may include a graphics processing unit (GPU), a neural processing unit (NPU), and / or a data processing unit (DPU), and may also be implemented as a separate chip that is physically independent from other components of the main processor 2100.

[0190] The memories 2200a and 2200b may be used as main memories of the system 2000 and may include volatile memories such as SRAM and / or DRAM. However, the present invention is not limited thereto, and the memories 2200a and 2200b may also include non-volatile memories such as flash memory, PRAM, and / or RRAM. The memories 2200a and 2200b may be implemented in the same package as the main processor 2100.

[0191] The memory devices 2300a and 2300b can be used as non-volatile memory devices that store data regardless of whether power is supplied, and can have a relatively large storage capacity compared to the memories 2200a and 2200b. The memory devices 2300a and 2300b may include memory controllers 2310a and 2310b and non-volatile memories (NVMs) 2320a and 2320b that store data under the control of the memory controllers 2310a and 2310b. The NVMs 2320a and 2320b may include flash memories having a 2-dimensional (2D) structure or a 3-dimensional (3D) V-NAND (vertical NAND) structure, but may also include other types of non-volatile memories such as PRAM and / or RRAM.

[0192] The storage devices 2300a and 2300b may be included in the system 2000, but may be physically separated from the main processor 2100, or may be implemented in the same package as the main processor 2100. In addition, the storage devices 2300a and 2300b may be solid-state devices (SSDs) or memory cards, and thus, the storage devices 2300a and 2300b may be detachably attached to other components of the system 2000 via an interface such as the connection interface 2480 to be described below. The storage devices 2300a and 2300b may be devices to which a standard protocol such as Universal Flash Storage (UFS), embedded MultiMediaCard (eMMC), or non-volatile memory express (NVMe) is applied, but are not necessarily limited thereto.

[0193] here, Figure 1 The memory device 20 may correspond to the memories 2200a and 2200b and / or the memory devices 2300a and 2300b.

[0194] The image capture device 2410 may capture still images or moving pictures and may include a still camera, a video camera, and / or a webcam.

[0195] The user input device 2420 may receive various types of data input from a user of the system 2000 and may include a touch pad, a keypad, a mouse, and / or a microphone.

[0196] The sensor 2430 may sense various types of physical quantities available from outside the system 2000 and convert the sensed physical quantities into electrical signals. The sensor 2430 may include a temperature sensor, a pressure sensor, an illumination sensor, a position sensor, an acceleration sensor, a biosensor, and / or a gyroscope sensor.

[0197] The communication device 2440 may transmit and receive signals to and from other devices outside the system 2000 according to various communication protocols. The communication device 2440 may include an antenna, a transceiver, and / or a modem.

[0198] Display 2450 and speaker 2460 may be used as output devices for outputting visual and auditory information, respectively, to a user of system 2000 .

[0199] The power supply device 2470 may convert power supplied from a battery embedded in the system 2000 and / or power supplied from an external power source and supply the converted power to components of the system 2000 .

[0200] The connection interface 2480 may provide a connection between the system 2000 and an external device that can be connected to the system 2000 and exchange data with the system 2000. The connection interface 2480 may be implemented as one of various interface protocols such as Advanced Technology Attachment (ATA), Serial ATA (SATA), external SATA (e-SATA), Small Computer Small Interface (SCSI), Serial Attached SCSI (SAS), Peripheral Component Interconnect (PCI), PCI Express (PCIe), NVM Express (NVMe), IEEE 1394, Universal Serial Bus (USB), Secure Digital (SD) card, MultiMediaCard (MMC), eMMC, Universal Flash Storage (UFS), embedded UFS (eUFS), and Compact Flash (CF) card interfaces.

[0201] While the inventive concept has been particularly shown and described with reference to embodiments thereof, it will be understood that various changes in form and details may be made therein without departing from the spirit and scope of the inventive concept as set forth in the following claims.

Claims

1. A memory device comprising a near memory device and a far memory device, in, The far memory device stores the compressed embedded vector, The near memory device comprises: a buffer configured to receive a request for pooling an embedding vector and output a hit signal or a miss signal in response to the request, wherein the hit signal includes the embedding vector corresponding to the request; an address calculation circuit configured to calculate a starting memory address and a memory size of the embedded vector corresponding to the request in response to the missing signal; a decoding circuit configured to obtain a compressed embedded vector corresponding to the request from the far memory device based on the starting memory address and the memory size, and output the embedded vector corresponding to the request by decoding the compressed embedded vector; and a pooling circuit configured to output a pooled embedding vector by performing a pooling operation on the embedding vector, wherein the pooling circuit obtains the embedding vector from the hit signal or obtains the embedding vector from the decoding circuit, and The buffer stores an embedded vector having an access frequency higher than a preset value among all embedded vectors.

2. The memory device of claim 1, wherein: Elements of the compressed embedded vector stored in the far memory device are quantized by retaining a predefined number of most significant bits from the mantissa bits and removing the remaining mantissa bits.

3. The memory device of claim 1, wherein: Elements of the compressed embedded vector stored in the far memory device are compressed by mapping the exponent bits to exponent bits having two bits or adding two bits having a first value between the sign bit and the exponent bit based on the bit value of the exponent bits.

4. The memory device of claim 1, wherein: The address calculation circuit is further configured to obtain values required to calculate the starting memory address and the memory size from a mapping table stored in the far memory device.

5. The memory device of claim 4, wherein: The address calculation circuit is further configured to calculate a starting memory address of the compressed embedded vector corresponding to the request based on a starting memory address of the compressed embedded table corresponding to the request and a value of the mapping table corresponding to the request.

6. The memory device of claim 4, wherein: The address calculation circuit is further configured to calculate a memory size of the compressed embedded vector corresponding to the request based on a starting memory address of the compressed embedded vector corresponding to the request and a starting memory address of a next embedded vector of the compressed embedded vector corresponding to the request.

7. The memory device of claim 1, wherein: The decoding circuit is further configured to replace the bit value of the exponent bit of the compressed embedded vector element with the bit value included in the exponent table based on the most significant bit and the second most significant bit of the exponent bits of the compressed embedded vector element, or to remove the most significant bit and the second most significant bit from the exponent bits of the compressed embedded vector element.

8. An electronic device comprising a near memory device and a far memory device, the electronic device comprising a processor configured to control the near memory device and the far memory device, in, The far memory device stores the compressed embedded vector, The near memory device comprises: a buffer configured to receive a request for pooling an embedding vector and output a hit signal or a miss signal in response to the request, wherein the hit signal includes the embedding vector corresponding to the request; an address calculation circuit configured to calculate a starting memory address and a memory size of the embedded vector corresponding to the request in response to the missing signal; a decoding circuit configured to obtain a compressed embedded vector corresponding to the request from the far memory device based on the starting memory address and the memory size, and output the embedded vector corresponding to the request by decoding the compressed embedded vector; and a pooling circuit configured to output a pooled embedding vector by performing a pooling operation on the embedding vector, wherein the pooling circuit obtains the embedding vector from the hit signal of the buffer or obtains the embedding vector from the decoding circuit, and The buffer stores an embedded vector having an access frequency higher than a preset value among all embedded vectors.

9. The electronic device according to claim 8, wherein: Elements of the compressed embedded vector stored in the far memory device are quantized by retaining a predefined number of most significant bits from the mantissa bits and removing the remaining mantissa bits.

10. The electronic device according to claim 8, wherein: Elements of the compressed embedded vector stored in the far memory device are compressed by mapping the exponent bits to exponent bits having two bits or adding two bits having a first value between the sign bit and the exponent bit based on the bit value of the exponent bits.

11. The electronic device according to claim 8, wherein: The address calculation circuit is further configured to obtain values required to calculate the starting memory address and the memory size from a mapping table stored in the far memory device.

12. The electronic device according to claim 11, wherein: The address calculation circuit is further configured to calculate a starting memory address of the compressed embedded vector corresponding to the request based on a starting memory address of the compressed embedded table corresponding to the request and a value of the mapping table corresponding to the request.

13. The electronic device according to claim 11, wherein: The address calculation circuit is further configured to calculate a memory size of the compressed embedded vector corresponding to the request based on a starting memory address of the compressed embedded vector corresponding to the request and a starting memory address of a next embedded vector of the compressed embedded vector corresponding to the request.

14. The electronic device according to claim 8, wherein: The decoding circuit is further configured to replace the bit value of the exponent bit of the compressed embedded vector element with the bit value included in the exponent table based on the most significant bit and the second most significant bit of the exponent bits of the compressed embedded vector element, or to remove the most significant bit and the second most significant bit from the exponent bits of the compressed embedded vector element.

15. A near memory device comprising: a buffer configured to receive a request for pooling an embedding vector and output a hit signal or a miss signal in response to the request, wherein the hit signal includes the embedding vector corresponding to the request; an address calculation circuit configured to calculate a starting memory address and a memory size of the embedded vector corresponding to the request in response to the missing signal; a decoding circuit configured to obtain a compressed embedded vector corresponding to the request from a far memory device based on the starting memory address and the memory size, and output the embedded vector corresponding to the request by decoding the compressed embedded vector; and a pooling circuit configured to output a pooled embedding vector by performing a pooling operation on the embedding vector, wherein the pooling circuit obtains the embedding vector from the hit signal of the buffer or obtains the embedding vector from the decoding circuit, and The buffer stores embedded vectors having an access frequency higher than a preset value among all embedded vectors.

16. The near memory device of claim 15, wherein: Elements of the compressed embedded vector stored in the far memory device are quantized by retaining a predefined number of most significant bits from the mantissa bits and removing the remaining mantissa bits.

17. The near memory device of claim 15, wherein: Elements of the compressed embedded vector stored in the far memory device are compressed by mapping the exponent bits to exponent bits having two bits or adding two bits having a first value between the sign bit and the exponent bit based on the bit value of the exponent bits.

18. The near memory device of claim 15, wherein: The address calculation circuit is further configured to obtain values required to calculate the starting memory address and the memory size from a mapping table stored in the far memory device.

19. The near memory device of claim 18, wherein: The address calculation circuit is further configured to calculate a starting memory address of the compressed embedded vector corresponding to the request based on a starting memory address of the compressed embedded table corresponding to the request and a value of the mapping table corresponding to the request.

20. The near memory device of claim 18, wherein: The address calculation circuit is further configured to calculate a memory size of the compressed embedded vector corresponding to the request based on a starting memory address of the compressed embedded vector corresponding to the request and a starting memory address of a next embedded vector of the compressed embedded vector corresponding to the request.

Images