Embedded query reasoning method for a memristor-based recommendation system accelerator

Abstract

The application provides an embedding query reasoning method of a recommendation system accelerator based on a memristor, for an input batch of sample data batch0, based on an embedding table, redundancy phenomena of corresponding index elements in the batch of sample data are analyzed respectively, repeated index elements are removed, and new non-redundant sample data batch1 is obtained; and a mapping between batch0 and batch1 is established; the non-redundant sample data batch1 is used to access a memory for embedding table query and reasoning, after the reasoning is completed, according to the mapping between the original sample data batch0 and the non-redundant sample data batch1, the reasoning result is copied and filled to the corresponding size of the original sample data batch0, and the final reasoning result corresponding to the original sample data batch0 is obtained. The application removes repeated query indexes in the input sample data, constructs new non-redundant data for reasoning, reduces redundant memristor access, and thus improves the overall reasoning performance of the system.

Description

Technical Field

[0001] This invention relates to the field of computer science and artificial intelligence, and in particular to an embedded query reasoning method based on a memristor-based recommender system accelerator. Background Technology

[0002] Personalized recommendation systems have been widely applied in industries such as search engines, social networks, e-commerce, video streaming, and advertising. Deep learning recommendation models (DLRMs) have become the mainstream recommendation system due to their high accuracy and ability to meet user needs. DLRMs use a hybrid model of multilayer perceptrons (MLPs) and embedding tables (EMBs) for personalized recommendations. MLPs handle dense input and perform final click-through rate prediction; EMBs transform high-dimensional sparse input into low-dimensional dense embedding vectors through lookup operations. Research shows that over 80% of DLRM execution time is spent on DRAM access and data communication, indicating a memory bottleneck.

[0003] To address this issue, numerous algorithm-hardware co-design techniques have been proposed. For handling matrix-vector multiplication in the core computation of DNNs, in-memory computing accelerators based on Resistive Random Access Memory (ReRAM) are more efficient than near-memory processing designs based on CMOS. However, directly porting recommender systems to ReRAM-based accelerators results in suboptimal hardware efficiency and performance. One reason is that a batch of input data in a recommender system often contains a large number of redundant vector queries, leading to many unnecessary memory accesses. Therefore, a technique is needed to eliminate these redundant vector queries from the data path of the ReRAM-based accelerator, thereby effectively improving the inference performance of recommender system models on ReRAM accelerators. Summary of the Invention

[0004] To address the issue of redundant memory accesses during inference in memristor-based ReRAM-based recommendation model accelerators, this invention proposes an embedding query inference method for memristor-based recommendation system accelerators. This method supports redundancy removal within batches to reduce redundant embedding vector queries in the data flow processing. For input sample data batch 0, unique query IDs are extracted from the embedding table to construct a new, non-redundant sample data batch 1. Inference is then performed using this new, non-redundant batch, reducing redundant memristor accesses and thus improving the overall inference performance of the system.

[0005] The specific technical solution adopted in this invention is as follows:

[0006] An embedded query reasoning method based on a memristor-based recommender system accelerator includes:

[0007] For a batch of sample data batch0, based on the embedding table, the redundancy of the corresponding index elements in the sample data of a batch is analyzed, duplicate index elements are removed, and a new batch of sample data without redundancy batch1 is obtained; and a mapping is established between the original sample data batch0 and the batch of sample data without redundancy batch1.

[0008] Based on the non-redundant sample data batch1, the memory is accessed for embedded table lookup and inference. After inference is completed, according to the mapping between the original sample data batch0 and the non-redundant sample data batch1, the inference result is copied and filled to the corresponding size of the original sample data batch0 to obtain the final inference result corresponding to the original sample data batch0.

[0009] Furthermore, based on the embedded table, the redundancy of corresponding index elements in the sample data of a batch is analyzed, and duplicate index elements are removed to obtain a new batch of non-redundant sample data, batch1. Specifically:

[0010] Based on each embedded table, obtain the corresponding index element in a batch of sample data, remove duplicate index elements, and obtain the non-redundant index element of the corresponding embedded table.

[0011] The non-redundant index elements of each embedded table are combined to obtain a new non-redundant sample data batch1.

[0012] Furthermore, based on the embedded table, the redundancy of corresponding index elements in the sample data of a batch is analyzed, and duplicate index elements are removed to obtain a new batch of non-redundant sample data, batch1. Specifically:

[0013] Based on each pair of embedded tables, obtain the index element of the corresponding pair of embedded tables in a batch of sample data, remove duplicate index elements, and obtain the non-redundant index element of the corresponding pair of embedded tables.

[0014] The non-redundant index elements of each pair of embedded tables are combined to obtain a new non-redundant sample data batch1;

[0015] Among them, the paired embedding table is obtained by calculating the similarity of the embedded vector access distribution data in the embedding table during the training process of the given recommendation system model, and then combining the two embedding tables with the highest similarity.

[0016] Furthermore, the similarity calculation method is Jenson-Shannon divergence, Kullback-Leibler divergence, etc.

[0017] Furthermore, the pairing embedding table is mapped to the same hardware unit tile when mapped to the memristor ReRAM.

[0018] Furthermore, the establishment of the mapping between the original sample data batch0 and the non-redundant sample data batch1 is specifically an input mapping index table; wherein, the size of the input mapping index table is the same as the size of the original sample data batch0; each element in the input mapping index table is the index of the corresponding index element in the new non-redundant sample data batch1.

[0019] An embedded query inference device for a memristor-based recommender system accelerator includes:

[0020] The redundancy removal and mapping module is used to analyze the redundancy of corresponding index elements in a batch of sample data (batch0) based on the embedding table, remove duplicate index elements, obtain a new batch of sample data (batch1) without redundancy, and establish a mapping between the original sample data (batch0) and the batch of sample data (batch1) without redundancy.

[0021] The query and inference module is used to access the memory based on the non-redundant sample data batch1 to perform embedded table queries and inference. After the inference is completed, according to the mapping between the original sample data batch0 and the non-redundant sample data batch1, the inference result is copied and filled to the corresponding size of the original sample data batch0 to obtain the final inference result corresponding to the original sample data batch0.

[0022] The beneficial effects of this invention are:

[0023] This invention proposes an embedding query inference method for a memristor-based recommender system accelerator. The method first supports redundancy removal within the batch to reduce redundant data flow processing of embedding vector queries. Then, for the input sample data batch0, the unique query IDs are extracted from the embedding table to construct a new, non-redundant sample data batch1. Finally, the new, non-redundant batch is used for inference, reducing redundant memristor access and thus improving the overall inference performance of the system. Attached Figure Description

[0024] Figure 1 This is a conventional method for processing the embedding layer of a ReRAM-based recommendation model;

[0025] Figure 2 This is a schematic diagram of the embedding layer processing method in step one of the embedding query reasoning method of the memristor-based recommendation system accelerator of the present invention;

[0026] Figure 3 This is a comparison chart showing the delay results of six models in this embodiment of the invention after being processed by the method of this invention and the conventional embedding layer processing method (baseline);

[0027] Figure 4 The graph shows the power consumption comparison results of six models in this embodiment of the invention after being processed by the method of this invention and the conventional embedding layer processing method (baseline). Detailed Implementation

[0028] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0029] This invention provides an embedded query reasoning method for a memristor-based recommender system accelerator, comprising the following steps:

[0030] Step 1: For the input batch of sample data batch0, based on the embedding table, analyze the redundancy of the corresponding index elements in the sample data of each batch, remove duplicate index elements, and obtain a new batch of sample data without redundancy batch1; and establish a mapping between the original sample data batch0 and the batch of sample data without redundancy batch1.

[0031] This step obtains batch1 of non-redundant sample data, which reduces redundant memristor accesses in step two, thereby reducing overall system inference latency and power consumption and improving overall system inference performance.

[0032] Specifically, in this step, for the input batch of sample data batch0, based on the embedding table, the redundancy of the corresponding index elements in the sample data of each batch is analyzed, and duplicate index elements are removed to obtain new non-redundant sample data batch1. Redundancy removal can be performed based on each embedding table, including the following sub-steps:

[0033] (1) Based on each embedding table, obtain the corresponding index element in a batch of sample data, remove duplicate index elements, and obtain the non-redundant index element of the corresponding embedding table; taking a given recommendation system model containing 26 embedding tables as an example, this step is performed 26 times.

[0034] (2) Combine the non-redundant index elements of each embedded table to obtain a new non-redundant sample data batch1.

[0035] As a preferred approach, this step can first pair the embedded tables based on their access patterns, and then perform redundancy removal on each pair of paired embedded tables. Since the access patterns of the paired embedded tables are similar, the redundancy removal efficiency is greatly improved without reducing the degree of redundancy removal, which is beneficial to improving the overall speed. Specifically, it includes the following sub-steps:

[0036] (1) Embedded table pairing based on access pattern

[0037] Current recommender system models typically have multiple embedding tables, each encoding embedding information for different feature domains. Each embedding table contains a varying number of embedding vectors, ranging from a few to tens of millions or even higher. The number of times each embedding vector within an embedding table is accessed during the training process of the recommender system model constitutes an access distribution for that embedding table. Therefore, based on the collected access distribution data of the embedding vectors within the embedding tables during the training process of a given recommender system model, the similarity of the access distributions of each embedding table is calculated. Similarity calculation methods can employ Jenson-Shannon (JS) divergence, Kullback-Leibler (KL) divergence, etc. In this embodiment, JS divergence is used as an example to represent the access pattern of the embedding table. Then, a matching method based on a greedy approach is used to obtain the pairing of embedding tables. Specifically, starting from the first embedding table, the system iterates through each embedding table, finding the embedding table with the closest JS divergence that has not yet been paired, and combining them for pairing. This results in multiple pairs of paired embedding tables.

[0038] (2) Batch data redundancy analysis

[0039] Based on each pair of embedded tables obtained in step (1), obtain the index element of the corresponding pair of embedded tables in a batch of sample data. Figure 1 This is a conventional embedding layer processing method based on ReRAM recommendation models. The sample data are index elements, which are decoded to obtain the corresponding embedding table query vectors. For example, in Figure 1 The input contains 8 samples: [[2,0],[2,0],[1,2],[0,1],[2,0],[1,2],[0,1],[2,0]]. This example uses a pair of paired embedding tables, so each sample contains only the query index corresponding to one pair of embedded tables. This shows obvious redundancy in the original sample data. By removing duplicate samples from the original data, we can obtain a new, non-redundant index element for the corresponding paired embedding table, such as... Figure 2 After removing redundancy, only three unique sample index elements remain: [[2,0],[1,2],[0,1]]. This indicates that during inference on ReRAM, only the index elements corresponding to these three samples need to be accessed (activated). It can be seen that assuming processing one sample requires one cycle, processing the original data requires eight cycles, while processing the deredundant data only requires three cycles, saving five cycles. Furthermore, taking a given recommender system model containing 26 embedding tables as an example, if all embedding tables are paired, this step can be performed 13 times.

[0040] (3) Combine the non-redundant index elements of each pair of embedded tables to obtain a new non-redundant sample data batch1.

[0041] Furthermore, as an optional implementation, when mapping to ReRAM, paired embedding tables are mapped to the same hardware unit tile. Since paired embedding tables have similar access patterns, mapping them to the same hardware unit tile facilitates embedding table lookups and inferences in subsequent steps.

[0042] Furthermore, in this step, a mapping is established between the original sample data batch0 and the non-redundant sample data batch1 for subsequent restoration. This mapping can be achieved through an input mapping index table, where the size of the input mapping index table is the same as the size of the original sample data batch0. Each element in the input mapping index table is the index of the corresponding index element in the new non-redundant sample data batch1. Figure 2 The eight samples shown correspond to the indices [0,2,1,0,2,1,0,0] in the input mapping index table.

[0043] Step 2: Based on the non-redundant sample data batch1, access the memory to perform embedded table lookup and inference. In order to ensure the consistency of the actual inference data size, after the inference is completed using batch1, according to the mapping between the original sample data batch0 and the non-redundant sample data batch1, copy and fill the inference result to the corresponding size of the original sample data batch0 to obtain the final inference result corresponding to the original sample data batch0. Figure 2 The three samples in batch 1, which contains no redundant sample data, will yield three results after embedding table lookup and inference. Each of the eight samples in batch 0 needs to correspond to an output result. Therefore, the corresponding results can be copied and expanded according to the index [0,2,1,0,2,1,0,0] in the input mapping index table to obtain the final inference result.

[0044] The present invention will be further described below with reference to specific embodiments, and specific experiments are as follows:

[0045] Experimental setup:

[0046] The experiment used the open-source ReRAM simulator NeuroSim-V1.3 (DNN+NeuroSim V1.3). The ReRAM configuration in this simulator is as follows:

[0047] (1) Cross array size: 128×128;

[0048] (2) Equipment process: Ag:Si device; 32nm process; each unit contains 1 bit of data;

[0049] (3) Tile size: 16 intersecting arrays; organized into a 4×4 matrix layout;

[0050] (4) Analog-to-digital converter precision: 5-bit;

[0051] (5) Digital-to-analog converter precision: 1-bit

[0052] Model configuration:

[0053] (1) The models used are shown in the table below, which are open source models from Facebook and Google respectively.

[0054] Table 1. Model and its structural representation

[0055] RM1 256-128-64 256-64-1 5 RM2 256-128-64 512-128-1 26 RM3 2560-512-64 512-128-1 5 RM4 512-256-64 512-512-256-1 26 DCN \ 512-128-1 26 DHEN 256-128-64 512-128-1 26

[0056] (2) Dataset:

[0057] • Criteo Kaggle Display Advertising (hereinafter referred to as Kaggle): It contains 13 continuous features and 26 categorical features; the Kaggle dataset is 10.5GB in size and contains more than 45 million click responses over 7 days;

[0058] • Criteo 1TB Logs (hereinafter referred to as Terabyte): Contains 13 continuous features and 26 categorical features; the Terabyte dataset is 1.3TB in size and contains more than 4.3 billion records of click feedback over 24 days;

[0059] (3) Batch size: 4096;

[0060] Final test results:

[0061] The embedded query inference method of the recommender system accelerator of this invention eliminates redundancy in the inference batch data of each model, thereby reducing the number of ReRAM accesses during inference on the ReRAM accelerator, thus improving overall latency and power consumption. Specifically, compared with pairing embedding tables and mapping paired embedding tables to the same tile, under a given batch size, the latency and power consumption of the six models used are optimized compared to the baseline as follows: Figure 3 , 4As shown, the embedded query inference method using the recommender system accelerator of this invention reduces overall latency and power consumption, and improves overall inference performance during the inference process of the recommender model based on the ReRAM accelerator.

[0062] Corresponding to the aforementioned embodiment of the embedded query reasoning method for a memristor-based recommender system accelerator, the present invention also provides an embodiment of an embedded query reasoning device for a memristor-based recommender system accelerator.

[0063] An embedded query inference device for a memristor-based recommender system accelerator includes:

[0064] The redundancy removal and mapping module is used to analyze the redundancy of corresponding index elements in a batch of sample data (batch0) based on the embedding table, remove duplicate index elements, obtain a new batch of sample data (batch1) without redundancy, and establish a mapping between the original sample data (batch0) and the batch of sample data (batch1) without redundancy.

[0065] The query and inference module is used to access the memory based on the non-redundant sample data batch1 to perform embedded table queries and inference. After the inference is completed, according to the mapping between the original sample data batch0 and the non-redundant sample data batch1, the inference result is copied and filled to the corresponding size of the original sample data batch0 to obtain the final inference result corresponding to the original sample data batch0.

[0066] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0067] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. An embedded query reasoning method for a recommender system accelerator based on memristors, characterized in that, include: For a batch of sample data batch0, based on the embedding table, the redundancy of the corresponding index elements in the sample data of a batch is analyzed, duplicate index elements are removed, and a new batch of sample data without redundancy batch1 is obtained; and a mapping is established between the original sample data batch0 and the batch of sample data without redundancy batch1. Based on the non-redundant sample data batch1, the memory is accessed for embedded table lookup and inference. After inference is completed, according to the mapping between the original sample data batch0 and the non-redundant sample data batch1, the inference result is copied and filled to the corresponding size of the original sample data batch0 to obtain the final inference result corresponding to the original sample data batch0. The recommender system model includes multiple embedding tables. The paired embedding table is obtained by calculating the similarity of the embedding vectors access distribution data within the embedding table during the training process of the given recommender system model, and then combining the two embedding tables with the highest similarity. The paired embedding table is mapped to the same hardware unit tile when mapped to the memristor ReRAM.

2. The method according to claim 1, characterized in that, The method involves analyzing the redundancy of corresponding index elements in a batch of sample data based on the embedded table, removing duplicate index elements, and obtaining a new batch of non-redundant sample data, batch1. Based on each pair of embedded tables, obtain the index element of the corresponding pair of embedded tables in a batch of sample data, remove duplicate index elements, and obtain the non-redundant index element of the corresponding pair of embedded tables. The new batch of non-redundant sample data, batch1, is obtained by combining the non-redundant index elements of each pair of embedded tables.

3. The method according to claim 2, characterized in that, The similarity calculation method is Jenson-Shannon divergence or Kullback-Leibler divergence.

4. The method according to claim 1, characterized in that, The specific method for establishing the mapping between the original sample data batch0 and the non-redundant sample data batch1 is to use an input mapping index table; wherein, the size of the input mapping index table is the same as the size of the original sample data batch0; each element in the input mapping index table is the index of the corresponding index element in the new non-redundant sample data batch1.

5. An embedded query inference device for a memristor-based recommender system accelerator, characterized in that, include: The redundancy removal and mapping module is used to analyze the redundancy of corresponding index elements in a batch of sample data (batch0) based on the embedding table, remove duplicate index elements, obtain a new batch of sample data (batch1) without redundancy, and establish a mapping between the original sample data (batch0) and the batch of sample data (batch1) without redundancy. The query and inference module is used to access the memory based on the non-redundant sample data batch1 to perform embedded table queries and inference. After the inference is completed, according to the mapping between the original sample data batch0 and the non-redundant sample data batch1, the inference result is copied and filled to the corresponding size of the original sample data batch0 to obtain the final inference result corresponding to the original sample data batch0. The recommender system model includes multiple embedding tables. The paired embedding table is obtained by calculating the similarity of the embedding vector access distribution data within the embedding table during the training process of the given recommender system model, and then combining the two embedding tables with the highest similarity. The paired embedding table is mapped to the same hardware unit tile when mapped to the memristor ReRAM.

Images