ReRAM recommendation system accelerator mapping method and system based on access pattern and entropy perception
By collecting access patterns and entropy awareness of the embedding table in the ReRAM accelerator, and performing clustering and quantization of the embedding vectors, the problem that the ReRAM accelerator cannot effectively map the embedding layer of the recommendation system is solved, which improves resource utilization, reduces energy consumption, and enhances the execution efficiency and accuracy of the recommendation system.
Patent Information
- Application Number
- CN202210266429.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-17
- Publication Date
- 2025-11-04
- Estimated Expiration
- 2042-03-17
AI Technical Summary
Existing ReRAM-based in-memory accelerators cannot effectively map the embedding layer of recommendation systems, resulting in low resource utilization and an inability to meet the high accuracy requirements of recommendation systems. They also suffer from memory bottlenecks and high energy consumption.
By collecting the access patterns and entropy awareness of the embedding table, the embedding vectors are clustered and quantized. The clustering mask mapping table is used to map the clustered embedding vectors and cluster centers onto the cross array of the ReRAM accelerator. The quantization bit width is determined according to the entropy value of the embedding table and the cross array configuration to optimize resource utilization.
It improves the resource utilization of the ReRAM accelerator, solves the memory bottleneck problem, reduces inference energy consumption, and improves the overall execution efficiency and accuracy of the recommendation system.
Smart Images
Figure CN114943338B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the field of computer science artificial intelligence, and particularly relates to a ReRAM recommendation system accelerator mapping method and system based on access patterns and entropy perception for solving the memory bottleneck problem in the training and inference process of a recommendation system. BACKGROUND
[0002] Recommendation systems are widely used in many fields such as social networks, e-commerce, search engines, etc. The embedding layer-based recommendation system model based on deep learning solution can well recommend according to the user's preference. Such a model includes a memory-intensive embedding layer and a computation-intensive neural network layer. Since the operation time of the embedding layer accounts for up to 80% of the overall recommendation model execution time of the recommendation system, the model faces a serious memory bottleneck during operation, thereby limiting the execution efficiency of the entire recommendation system.
[0003] To solve the above problems, the emerging ReRAM-based memory-compute integrated accelerator can use cross-array in-place computation and high parallelism to accelerate model execution and reduce system energy consumption. The cross-array resource of the ReRAM memory-compute integrated accelerator is limited and cannot directly map the huge embedding layer of the recommendation system. At present, there is no compression scheme for the embedding layer of the recommendation system designed for the memory-compute integrated accelerator. If the configuration parameters of the cross-array hardware and the importance of the access pattern and the embedding table in the recommendation system load are not considered for direct clustering and quantization, the resource utilization rate of the memory-compute integrated accelerator will be low, and the high accuracy requirement of the recommendation system model cannot be met. SUMMARY
[0004] To solve the problem of low performance and high energy consumption of the embedding layer-dominated recommendation system model in the training and inference process, the application proposes a ReRAM recommendation system accelerator mapping method and system based on embedding table access patterns and entropy perception with high resource utilization.
[0005] The technical scheme adopted by the application is:
[0006] A ReRAM recommendation system accelerator mapping method based on access patterns and entropy perception, comprising:
[0007] Collecting the access pattern of the embedding vector of each embedding table of the recommendation system model in the training process to obtain the access frequency of the embedding vector;
[0008] Clustering the embedding vectors of each embedding table; at the same time, recording the mapping relationship between all embedding vectors in the embedding table and their clustered cluster centers to obtain a cluster mask mapping table; wherein, when clustering, the weight is set according to the access frequency of the embedding vector, so that the embedding vector with high access frequency has a higher probability of becoming a cluster center;
[0009] Mapping the clustered cluster center embedding vector and the cluster mask mapping table in each embedding table to the cross array of the ReRAM recommendation system accelerator to complete the mapping of the embedding table.
[0010] Further, it further comprises the steps of quantizing the bit width of the embedding vector by using the entropy value of the embedding table and the configuration of the cross array:
[0011] According to the configuration of the cross array and the feature dimension of each embedding vector of the embedding table, the minimum number of quantization bits in the quantization bit width candidate set of each embedding table is calculated;
[0012] According to the minimum number of quantization bits, the quantization bit width candidate set of each embedding table is determined;
[0013] According to the entropy value of each embedding table calculated according to the embedding vector access sequence, and the quantization bit width candidate set and the entropy value of each embedding table, the quantization bit width of the embedding vector of each embedding table is determined, wherein the greater the entropy value, the smaller the quantization bit width of the embedding vector of the embedding table.
[0014] Further, the minimum number of quantization bits is specifically: Wherein, xb is the cross array size of the ReRAM accelerator, N is the number of bits that can be mapped by each unit of the cross array, and D is the hidden feature dimension contained in each embedding vector of the embedding table.
[0015] Further, according to the minimum number of quantization bits, the quantization bit width candidate set of each embedding table is determined, specifically: if the minimum number of quantization bits is not greater than 2, the quantization bit number candidate set is {2, 4, 8}; if the minimum number of quantization bits is not greater than 4, the quantization bit number candidate set is {4, 8}; otherwise, the quantization bit number candidate set is {8}.
[0016] Further, it further comprises:
[0017] The embedding table after clustering is parameterized by retraining.
[0018] Further, when clustering, the number of cluster centers nc is determined by the following method:
[0019] nc=ceil(m / r / xb)*xb
[0020] r represents the cluster center scaling coefficient, m is the number of embedding vectors of the embedding table, xb is the cross array size of the ReRAM accelerator, and ceil() is the ceiling function.
[0021] A ReRAM recommendation system accelerator mapping system based on access mode and entropy perception, comprising:
[0022] a sampling analysis module for collecting the access mode of the embedding vectors of each embedding table by the recommendation system model during the training process to obtain the access frequency of the embedding vectors;
[0023] a clustering module for clustering the embedding vectors of each embedding table; and recording the mapping relationship between all embedding vectors in the embedding table and the cluster centers after clustering to obtain a cluster mask mapping table; wherein, during clustering, weights are set according to the access frequency of the embedding vectors, so that the embedding vectors with high access frequency have a higher probability of becoming cluster centers;
[0024] a ReRAM mapping data flow module for mapping the cluster center embedding vectors after clustering and the cluster mask mapping table in each embedding table to the cross array of the ReRAM recommendation system accelerator to complete the mapping of the embedding table.
[0025] Further, an entropy-aware quantization module is further included for utilizing the entropy value of the embedding table and the configuration of the cross array to decide the quantization bit width of the embedding vectors during mapping.
[0026] The present application has the following beneficial effects: the present application performs sampling analysis on the access mode of the data set of the recommendation system load before mapping the recommendation system, fully utilizes the access mode of the embedding table to cluster the embedding vectors, utilizes the entropy value of the embedding table to perform hybrid quantization on the embedding table mapped on the ReRAM accelerator, so that the ReRAM accelerator executes the recommendation system model on the limited cross array resources, solves the memory bottleneck problem, thereby accelerating the entire inference process and reducing the inference energy consumption. BRIEF DESCRIPTION OF DRAWINGS
[0027] Figure 1 (a) and (b) are embedding vector access mode sampling diagrams on Kaggle data sets
[0028] Figure 2 Overall component flowchart 。 DETAILED DESCRIPTION
[0029] The present application proposes a ReRAM recommendation system accelerator mapping method based on access mode and entropy awareness, comprising:
[0030] collecting the access mode of the embedding vectors of each embedding table by the recommendation system model during the training process to obtain the access frequency of the embedding vectors;
[0031] clustering the embedding vectors of each embedding table; and recording the mapping relationship between all embedding vectors in the embedding table and the cluster centers after clustering to obtain a cluster mask mapping table; wherein, during clustering, weights are set according to the access frequency of the embedding vectors, so that the embedding vectors with high access frequency have a higher probability of becoming cluster centers;
[0032] The clustered cluster center embedding vector and the cluster mask mapping table in each embedding table are mapped to the cross array of the ReRAM recommendation system accelerator to complete the mapping of the embedding table.
[0033] The application fully utilizes the access mode of the embedding vector in each embedding table in the recommendation system model, which has a very obvious "long tail phenomenon", as shown in the figure, that is, most accesses are mainly concentrated on a small number of embedding vectors. Figure 1 The access mode of the embedding table is fully utilized to cluster the embedding vector, and the clustered cluster center embedding vector and the cluster mask mapping table are mapped to the accelerator, so that the ReRAM accelerator executes the recommendation system model on the limited cross array resource, solves the memory bottleneck problem, thereby accelerating the entire inference process and reducing the inference energy consumption.
[0034] Wherein, when clustering, the number of cluster centers nc needs to meet the memory requirement of the ReRAM accelerator, which is determined by the following method:
[0035] nc=ceil(m / r / xb)*xb
[0036] r represents the cluster center scaling coefficient, m is the number of embedding vectors of the embedding table, xb is the cross array size of the ReRAM accelerator, and ceil() is the ceiling function.
[0037] Further, as shown in Table 1, the greater the entropy value of the access sequence of each embedding table in the recommendation system model, the smaller the influence on the precision, that is, fewer quantization bit widths can be used for quantization. The method also includes the steps of determining the quantization bit width of the embedding vector when mapping according to the entropy value of the embedding table and the configuration of the cross array: calculating the minimum number of quantization bits in each quantization bit width candidate set of each embedding table according to the configuration of the cross array and the feature dimension of each embedding vector of the embedding table;
[0038] According to the minimum quantization bit number, the quantization bit width candidate set of each embedding table is determined;
[0039] According to the access frequency sequence, the entropy value of each embedding table is calculated, and then the quantization bit width of the embedding vector of each embedding table is determined according to the quantization bit width candidate set and the entropy value of each embedding table, wherein the greater the entropy value, the smaller the quantization bit width of the embedding vector of the embedding table.
[0040] The entropy value of the embedding table is used to perform hybrid quantization on the embedding table mapped on the ReRAM accelerator, further solving the memory bottleneck problem.
[0041] Table 1. Sensitivity test table of embedding tables with different access sequence entropy values in DLRM model on precision.
[0042] Quantization scheme (x-y) Accuracy (%) Loss of precision 32-32 (original) 78.978 / 32-4 78.975 0.003 4-32 78.978 0 32-2 78.824 0.154 2-32 78.954 0.024 8-2 78.823 0.155 2-8 78.954 0.024 4-2 78.817 0.161 2-4 78.953 0.025
[0043] The DLRM model has 26 embedding tables, which are arranged in descending order of entropy. The first half of the embedding tables has a quantization bit width of x, and the second half of the embedding tables has a quantization bit width of y. The results in the table show that the greater the entropy value of the embedding table, the less sensitive it is to precision, and the quantization bit width can be selected to be smaller. For example, the "2-4" scheme has higher recommended precision than the "4-2" scheme.
[0044] The application will be described in detail below with reference to the accompanying drawings.
[0045] Step 1: Collect the access mode of each embedding vector of the embedding table in the training process of the recommendation system model: use the training set of the data set to count the access times of the embedding vector of the embedding table to obtain the access frequency fre of the embedding vector and the embedding vector access sequence Acess_list sorted according to the access time.
[0046] Step 2: Cluster the embedding vectors of each embedding table; record the mapping relationship between all embedding vectors in the embedding table and their cluster centers after clustering, obtain the cluster mask mapping table; wherein, the clustering is performed according to the size of the cross array and the access frequency distribution mode of the embedding vector, and the weight is set according to the access frequency of the embedding vector, so that the embedding vector with high access frequency has a higher probability of becoming a cluster center; thereby reducing the cross array resources required for mapping under the requirement of recommendation accuracy.
[0047] Specifically, the clustering based on the similarity of embedding vectors is performed using the access frequency distribution characteristics of the embedding table and the size of the cross array. The main steps are as follows:
[0048] Step 2.1: Assume that the cross array size of the ReRAM accelerator is xb, the access sequence of each embedding table is Access_list, the number of cluster centers of each embedding table is nc, the cluster center sequence and the cluster mask mapping table (which records the mapping relationship between the embedding vectors after clustering and the corresponding cluster centers) of each embedding table are C and M respectively, and the number of the embedding table is i;
[0049] Step 2.2: Empty the cluster center sequence of each embedding table, i.e. C = []; empty the cluster mask mapping table of each embedding table, i.e. M = [];
[0050] Step 2.3: Count the number of embedding vectors of the i-th embedding table as m, and count the access frequency sequence W of the embedding vectors of the i-th embedding table;
[0051] Step 2.4: Calculate the number of cluster centers nc of the embedding table according to the cross matrix size xb and the number of embedding vectors m, nc = ceil(m / r / xb)*xb; r represents the cluster center scaling coefficient.
[0052] Step 2.5: According to the access frequency sequence W (as weight) and the number of cluster centers nc of the embedding table as input parameters, the weighted_kmeans() function provided by PyTorch is used for clustering, and the cluster center sequence C and the cluster mask mapping table M are updated;
[0053] Step 2.6: Go to the next embedding table, i.e., embedding table number i = i + 1; go to step 2.2 until all embedding tables are traversed;
[0054] Step 2.7: Parameter tuning is performed on the clustered embedding table through retraining to improve recommendation accuracy.
[0055] Step 3: According to the collected embedding vector access frequency and access sequence, the mixed quantization bit width strategy decision is made for the re-optimized embedding layer.
[0056] Specifically, the entropy value of the embedding vector access sequence of the embedding table and the configuration of the cross array are used to decide the quantization bit width. The main steps are as follows:
[0057] Step 3.1: Assuming that the cross array size of the ReRAM accelerator is xb, each cell of the cross array can map N bits, each vector of the embedding table contains D hidden features, the embedding vector access frequency sequence W of the embedding table, and the number of the embedding table is i;
[0058] Step 3.2: Calculate the minimum quantization bit number Bottom candidate in the quantization bit width candidate set. That is, calculate
[0059] Step 3.3: Determine the quantization bit width candidate set S. If Bottom candidate is not greater than 2, then the quantization bit number candidate set is {2, 4, 8}; otherwise, if Bottom candidate is not greater than 4, then the quantization bit number candidate set is {4, 8}; otherwise, the quantization bit number candidate set is {8};
[0060] Step 3.4: According to the embedding vector access frequency sequence W, the current vector frequency list P is obtained, i.e., P = W;
[0061] Step 3.5: According to the access frequency sequence W, the entropy value of the i-th embedding table is calculated. That is, E = -∑ j P(j)log2P(j), where P(j) is the access frequency of the j-th embedding vector;
[0062] Step 3.6: Store the embedding table number i and the entropy value E in the key-value pair list E_list;
[0063] Step 3.7: go to the next embedding table, i.e. embedding table number i = i + 1; go to step 3.2 until all embedding tables are traversed;
[0064] Step 3.8: reorder the key-value pairs in the list E_list in descending order of the entropy value in each key-value pair to obtain a new E_list_orderd;
[0065] Step 3.9: decide the quantization bit width of each embedding table according to the ordered key-value list E_list_orderd and the candidate set S of quantization bit widths, and the decision principle is that the embedding table with a larger entropy value has a smaller quantization bit width.
[0066] Step 4: map the embedding layer to the cross array of the ReRAM recommendation system accelerator. According to the quantization decision of the embedding table, mixed-precision quantization is performed on the ReRAM accelerator.
[0067] Finally, during the recommendation system runtime on the mapped accelerator, the mask mapping table is used to complete the index conversion of the input embedding vector and the actually mapped embedding vector. The vectors involved in the input are filtered out to perform embedding operations and complete the recommendation system reasoning.
[0068] Corresponding to the foregoing embodiment of the ReRAM recommendation system accelerator mapping method based on access mode and entropy perception, the application also provides an embodiment of a ReRAM recommendation system accelerator mapping system based on access mode and entropy perception.
[0069] Referring to Figure 2 The ReRAM recommendation system accelerator mapping system based on access mode and entropy perception provided by the embodiment of the application comprises a sampling analysis module, a clustering module, an entropy perception quantization module and a ReRAM mapping data flow module. Wherein:
[0070] The sampling analysis module is used to acquire and analyze the access information of the embedding table, i.e. the access frequency of the embedding vectors in each embedding table and the embedding table access sequence.
[0071] The clustering module is responsible for reducing the number of embedding vectors for each embedding table: clustering the embedding vectors of each embedding table; at the same time, recording the mapping relationship between all embedding vectors in the embedding table and the cluster centers after clustering to obtain a cluster mask mapping table; wherein, during clustering, the access frequency of the embedding vectors is set as the weight, so that the embedding vectors with high access frequency have a higher probability of becoming cluster centers.
[0072] The entropy perception quantization module is responsible for selecting the optimal quantization bit width for each embedding table that meets the resource size and recommendation accuracy requirements of the cross array.
[0073] The ReRAM mapping data flow module is responsible for executing the embedding layer of the recommendation system on the memory-computing integrated accelerator: mapping each embedding table, the clustered cluster center embedding vector and the cluster mask mapping table to the cross array of the ReRAM recommendation system accelerator to complete the mapping of the embedding table.
[0074] The implementation processes of the functions and roles of the various modules in the system are specifically described in the implementation processes of the corresponding steps in the above method, and will not be described here.
[0075] Obviously, the above embodiments are only examples for clear illustration, and are not limitations on the embodiments. Based on the above description, other different forms of changes or variations can also be made by those of ordinary skill in the art. All embodiments do not need to be exhausted here. The obvious changes or variations derived therefrom are still within the protection scope of the present application.
Claims
1. An access pattern and entropy-aware based ReRAM recommendation system accelerator mapping method, characterized in that, The method comprises the following steps: Collecting the access mode of each embedding table in the training process of the recommendation system model, obtaining the access frequency of the embedding vector; Clustering the embedding vectors of each embedding table; at the same time, recording the mapping relationship between all embedding vectors in the embedding table and the cluster center after clustering, obtaining the cluster mask mapping table; wherein, when clustering, the weight is set according to the access frequency of the embedding vector, so that the embedding vector with high access frequency has a higher probability of becoming a cluster center; Mapping the cluster center embedding vector and the cluster mask mapping table in each embedding table to the cross array of the ReRAM recommendation system accelerator to complete the mapping of the embedding table; Further comprising a step of utilizing the entropy value of the embedding table and the configuration of the cross array to decide the quantization bit width of the embedding vector when mapping: According to the configuration of the cross array and the feature dimension of each embedding vector of the embedding table, the minimum quantization bit number in the quantization bit width candidate set of each embedding table is calculated; According to the minimum quantization bit number, the quantization bit width candidate set of each embedding table is determined; According to the access frequency of the embedding vector, the entropy value of each embedding table is calculated, and then according to the quantization bit width candidate set and the entropy value of each embedding table, the quantization bit width of the embedding vector of each embedding table is determined, wherein the larger the entropy value is, the smaller the quantization bit width of the embedding vector of the embedding table is.
2. The method of claim 1, wherein, The minimum quantization bit number is specifically: Wherein, xb is the cross array size of the ReRAM accelerator, N is the number of bit positions that can be mapped by each unit of the cross array, and D is the hidden feature dimension contained in each embedding vector of the embedding table.
3. The method of claim 1, wherein, According to the minimum quantization bit number, the quantization bit width candidate set of each embedding table is determined, specifically: if the minimum quantization bit number is not greater than 2, the quantization bit number candidate set is {2, 4, 8}; If the minimum quantization bit number is not greater than 4, the quantization bit number candidate set is {4, 8}; otherwise, the quantization bit number candidate set is {8}.
4. The method of claim 1, wherein, Further comprising: Parameter optimization is performed on the embedding table after clustering through retraining.
5. The method of claim 1, wherein, When clustering, the number of cluster centers nc is determined by the following method: nc=ceil(m / r / xb)*xb r represents the cluster center scaling coefficient, m is the number of embedding vectors of the embedding table, xb is the size of the cross array of the ReRAM accelerator, and ceil() is the ceiling function.
6. An access pattern and entropy-aware based ReRAM recommendation system accelerator mapping system, comprising: The method comprises the following steps: A sampling analysis module is configured to collect the access mode of each embedding table in the training process of the recommendation system model, and obtain the access frequency of the embedding vector; A clustering module is configured to cluster the embedding vectors of each embedding table; at the same time, record the mapping relationship between all embedding vectors in the embedding table and the cluster center after clustering, and obtain the cluster mask mapping table; wherein, when clustering, the weight is set according to the access frequency of the embedding vector, so that the embedding vector with high access frequency has a higher probability of becoming a cluster center; A ReRAM mapping data flow module is configured to map the cluster center embedding vector and the cluster mask mapping table in each embedding table to the cross array of the ReRAM recommendation system accelerator to complete the mapping of the embedding table; Further comprising an entropy-aware quantization module configured to utilize the entropy value of the embedding table and the configuration of the cross array to decide the quantization bit width of the embedding vector when mapping: According to the configuration of the cross array and the feature dimension of each embedding vector of the embedding table, the minimum quantization bit number in the quantization bit width candidate set of each embedding table is calculated; According to the minimum quantization bit number, the quantization bit width candidate set of each embedding table is determined; The entropy value of each embedding table is calculated according to the access frequency of the embedding vector, and the quantization bit width of the embedding vector of each embedding table is determined according to the candidate set of the quantization bit width of each embedding table and the entropy value, wherein the greater the entropy value is, the smaller the quantization bit width of the embedding vector of the embedding table is.