An in-memory processing based personalized recommendation system

Abstract

The application discloses an in-memory processing-based personalized recommendation system, and relates to the field of personalized recommendation.The system comprises a bottom full-connection engine, an embedding engine, a buffer, a top full-connection engine and a central processing unit.The embedding engine comprises an embedding module and a prediction module.The embedding module is composed of a ReRAM cross array, which is denoted as an embedding array, and each row of the embedding array stores a user interaction vector.The prediction module is composed of a group of vertically arranged ReRAM cross arrays, which is denoted as a prediction array, and each column of the prediction array is connected to the corresponding row in the embedding array.The embedding engine in the in-memory processing-based personalized recommendation system provided by the application sets the embedding module and the prediction module, utilizes the parallelism and high energy efficiency of ReRAM, reduces the bandwidth requirement, accelerates the embedding calculation of the embedding engine, and thus the computing speed of the personalized recommendation system is improved, and the efficiency and accuracy of the recommendation result are improved.

Description

Technical Field

[0001] This invention relates to the field of personalized recommendation technology, and in particular to a personalized recommendation system based on in-memory processing. Background Technology

[0002] Deep learning-based personalized recommendation models (DLRM) are widely used in various industries, such as Taobao product recommendations, Amazon book recommendations, and JD.com product recommendations. This model can recommend relevant products to users based on their basic information and preferences. However, the embedding layer's computation becomes a bottleneck due to the large amount of data required to store in its embedding table and the computational sparsity of the embedding operations.

[0003] Near-data processing architecture (NDP) accelerates embedding operations by reducing data flow compared to traditional memory architectures. TensorDIMM and RecNMP utilize NDP architecture, deploying near-memory processing units around dual in-line memory modules (DIMMs), increasing the accelerator's memory space. The TRiM algorithm also employs NDP architecture, further accelerating embedding operations by leveraging a tree-like interconnect topology based on DIMM memory.

[0004] However, the NDP method still requires the transmission of large amounts of data, which limits the overall computational parallelism and throughput of the computer. At the same time, it consumes a significant amount of energy, thus being constrained by energy efficiency requirements.

[0005] The REREC accelerator for personalized recommendation systems has several limitations that hinder its widespread adoption. It can only operate under low loads, and the size of the embedding table far exceeds the storage capacity of a typical ReRAM chip. Secondly, it only supports two specific recommendation systems that use inner product operations as the feature interaction method. However, mainstream DLRM embedding operations widely employ a clustering-reduction pattern, exhibiting irregular and sparse computational characteristics, posing a greater challenge to designing acceleration systems on ReRAM. Summary of the Invention

[0006] The purpose of this invention is to provide a personalized recommendation system based on in-memory processing, in order to solve the problems of slow calculation speed, inaccurate recommendation results and low recommendation efficiency of existing recommendation models.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] A personalized recommendation system based on in-memory processing includes: a bottom fully connected engine, an embedded engine, a buffer, a top fully connected engine, and a central processing unit (CPU). The input data of the bottom fully connected engine is a user feature vector; the input data of the embedded engine is a user interaction vector; the input of the buffer is connected to the outputs of both the bottom fully connected engine and the embedded engine, and the buffer is used to cache the output data of the bottom fully connected engine and the embedded engine; the input of the top fully connected engine is connected to the output of the buffer, and the top fully connected engine is used to calculate the data cached in the buffer to obtain recommendation results; the CPU is connected to the bottom fully connected engine, the embedded engine, the buffer, and the top fully connected engine, and is used to control the operation of the bottom fully connected engine, the embedded engine, the buffer, and the top fully connected engine respectively.

[0009] The embedding engine includes an embedding module and a prediction module; the embedding module consists of a ReRAM cross array, which is represented as an embedding array, and each row of the embedding array stores a user interaction vector; the prediction module consists of a set of vertically arranged ReRAM cross arrays, which is represented as a prediction array, and each column of the prediction array is connected to the corresponding row in the embedding array.

[0010] Optionally, the embedding engine also includes a dual in-line memory module.

[0011] Optionally, the embedding engine processes the user interaction vector in three stages: a prediction stage, a jump stage, and an execution stage.

[0012] Optionally, the embedding module is used to perform the execution phase, and the prediction module is used to perform the prediction phase and the execution phase.

[0013] Optionally, the bottom fully connected engine, the top fully connected engine, and the embedded engine are all provided with special function modules; the special function modules include: shift and addition units, scalar arithmetic units, and logic units.

[0014] Optionally, the special function module in the embedded engine is further provided with a non-zero detection unit, which is used to perform the jump phase.

[0015] Optionally, a delayed matching pipeline mechanism is embedded when the embedding engine processes the user interaction vector; the delayed matching pipeline mechanism is used to reorder the prediction phase, the jump phase, and the execution phase.

[0016] Optionally, an offline strategy is embedded when the embedding engine processes the user interaction vectors; the offline strategy reorders the user interaction vectors stored in the embedding array and the prediction array.

[0017] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0018] The embedding engine in the personalized recommendation system based on in-memory processing provided by this invention is equipped with an embedding module and a prediction module. By utilizing the parallelism and high energy efficiency of ReRAM, the bandwidth requirement is reduced, and the embedding calculation of the embedding engine is accelerated, thereby improving the calculation speed of the personalized recommendation system and improving the efficiency and accuracy of the recommendation results. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the structure of the personalized recommendation system based on in-memory processing provided by the present invention;

[0021] Figure 2 This is a diagram illustrating the data flow.

[0022] Figure 3 This is a schematic diagram of a delayed matching pipeline mechanism. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The purpose of this invention is to provide a personalized recommendation system based on in-memory processing. By optimizing the embedding engine, its embedding computation efficiency is improved, thereby accelerating the computation speed of the personalized recommendation system and improving the accuracy of the output results.

[0025] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Example 1

[0027] like Figure 1 As shown, the personalized recommendation system based on in-memory processing provided by the present invention includes: a bottom fully connected engine 1, an embedded engine 2, a buffer 3, a top fully connected engine 4, and a central processing unit 5.

[0028] User feature vectors are processed as dense input data into the bottom fully connected engine 1; user feature vectors include information such as user category, user age, and place of residence.

[0029] User interaction vectors are fed into the embedding engine 2 as sparse inputs for processing; user interaction vectors include information such as user preferences and user purchase history.

[0030] The processing results from the bottom fully connected engine 1 and the embedded engine 2 finally enter the buffer 3 for the next step of processing.

[0031] The input of the top fully connected engine 4 is connected to the output of the buffer 3 to perform calculations on the data cached in the buffer 3 and obtain the recommendation result.

[0032] The bottom fully connected engine 1, the top fully connected engine 4, and the embedded engine 2 are all equipped with special function modules (SFUs); the special function modules include shift and addition units, scalar arithmetic units, and logic units, in order to further process data.

[0033] The central processing unit 5 is responsible for loading input data and generating instructions to control the entire computing process.

[0034] The sparsity in the embedding operations of Embedding Engine 2 leads to inefficiency. Typically, each embedding operation requires one clock cycle to complete; a cycle in which computation is performed is denoted as CU, and the uncomputed cycle as CUs. CUs constitute a very high proportion, resulting in low computational efficiency. Therefore, to address the computational inefficiency caused by sparsity, an operational strategy was designed in the embedding operations to improve computational efficiency and reduce power consumption. The operational strategy consists of three phases:

[0035] In the prediction phase, predict useless CUs.

[0036] During the jump phase, the reserved CUs are moved forward to the front of the timeline.

[0037] During the execution phase, the reserved CUs are executed sequentially.

[0038] To implement the above strategy, an embedding module and a prediction module are set up in the embedding engine 2. The embedding module consists of a ReRAM cross array, represented as an embedding array, with each row storing a user interaction vector, mainly used in the execution phase. The prediction module consists of a set of vertically arranged ReRAM cross arrays, represented as a prediction array, with each part connected to a row in the embedding module, mainly used in the prediction and execution phases. The prediction module has a prediction mode and a read mode for the prediction and execution phases, respectively. The prediction array deploys selectable DACs in each row and column. An optimized non-zero detection module (LNZD) is also set up in the SFU for the jump phase. The input vector enters the prediction module as the first data stream. The CU state vector enters the LNZD as the second data stream. After generating one-hot encoding, it enters the embedding module as the third data stream.

[0039] like Figure 2 As shown, the data flow under the operating strategy is as follows:

[0040] Prediction Phase: The DAC in each row is activated to achieve prediction mode. The CPU 5 inputs a "1", and the prediction array sums the values ​​in each column. If any column's result is greater than or equal to 1, it indicates that at least one item has interacted with the user. At this point, the bottom amplifier (SA) outputs a "1" bit, reserving the corresponding CU in one row of the cross array. Otherwise, it outputs a "0" bit as a signal to skip the CU. The outputs of all columns are combined into a CU state vector, which is further used in the next phase.

[0041] Skip Phase: The CU state vector is input into the LNZD, and the offset of each non-zero feature vector is output cyclically. The central controller converts these offsets into one-hot encodings, representing the global index of an item that the user has interacted with at least once. The execution phase searches for the corresponding interactive item. Since the CU state vector is relatively long, it is segmented into shorter vectors to reduce the hardware burden.

[0042] Execution phase: The prediction array operates in read mode, with the DAC enabled for each column. Once a column is correctly read by the SA, the output term is added to a row of the embedding array, and the feature vector of the interaction term is selected for aggregation and reduction operations.

[0043] Embedded Engine 2 also includes Dual In-line Memory Modules (DIMMs), which can help expand storage capacity to store the entire embedding table. As many user interaction vectors as possible are cached on the highest-frequency ReRAM, and the remaining user interaction vectors are stored on the DIMMs.

[0044] During the embedding operation, the system loads the user interaction vectors from ReRAM and DIMMs, and obtains the final embedding result through SFU output. This operating strategy buffers some of the computation results. Then, the controller records the original user interaction vectors and selects from them, finally merging the results to obtain the embedding result for each user.

[0045] Writing user interaction vectors into ReRAM results in significant latency, making the prediction phase much longer than the execution and jump phases. Therefore, this invention designs a latency-matching pipeline mechanism as a three-phase rearrangement method to address this issue, such as... Figure 3 As shown, the prediction array resources are tripled, and three batches of data are written in parallel to three rows of the prediction array. The prediction and execution phases of the three batches are rearranged. The first batch forms the first-level pipeline, and the following two batches form the second-level pipeline. During the prediction phase, the skip and execution phases are rearranged, allowing parallel entry into the skip and execution phases. Simultaneously, a system-level pipeline is utilized for computation on ReRAM. This reduces latency, ensuring that the DIMM bandwidth is no longer a bottleneck in the design.

[0046] Running strategies can improve the computational efficiency of ReRAM, but this can lead to some ReRAM being idle. To utilize these idle resources, this invention designs an offline strategy that balances the workload, solving the problem of unbalanced ReRAM computation when executing running strategies. The offline strategy algorithm is as follows:

[0047] Algorithm: Heuristic K-partitioning offline strategy

[0048] Input: number of ReRAMs k, access frequencies of n user vectors in descending order;

[0049] Output: Vector allocation results for k ReRAMs;

[0050] Steps: 1. Create k vectors, each containing two values: the access frequency of ReRAM (initially 0) and a list of user interaction vectors (initially empty). Then, build a min-heap sorted by the access frequency of these k vectors.

[0051] 2. Loop n times, each time performing the following: extract the top vector of the min-heap; add the access frequency of the top vector to the access frequency of the user vector in the i-th loop; add the user vector in the i-th loop to the user interaction vector list of the top vector; then add the top vector back to the min-heap and adjust the heap.

[0052] This offline strategy reorders the items stored in ReRAM so that each memory location has an equal probability of activation. The algorithm is a linear K-partition problem, solved by a greedy heuristic.

[0053] The advantages of this invention compared to existing methods are as follows:

[0054] (1) The demand for DIMM bandwidth has been significantly reduced.

[0055] (2) Embedding operations are performed using a parallel, energy-efficient ReRAM cross array, providing more computational space for the data.

[0056] (3) This invention utilizes the parallelism and high energy efficiency of ReRAM to reduce bandwidth requirements. This invention also proposes a three-stage operation strategy to optimize throughput, an offline strategy based on K-partition heuristic to achieve load balancing, and a delay matching pipeline to improve throughput.

[0057] In existing methods, operands must be loaded from memory into the cache for computation. PIMPR, however, can load directly from the DIMM to the SFU. This data path avoids writing to ReRAM, reducing latency and power consumption.

[0058] The DLRM used in the experiments contained 8 embedding tables and 6 fully connected layers, with a batch size of 128. Six datasets were selected, and four of them were merged into a single ultra-large dataset to simulate the heavy workload environment of the recommender system. Compared with TensorDIMMx2 and TRiM, PIMPR achieved an average speedup of 6.72 and 1.79 times, and 36.2 and 5.1 times, respectively.

[0059] PIMPR only needs to load low-access items from the DIMM, greatly reducing bandwidth pressure. The average latency of this system is 0.29 times that of computation on the ReRAM chip, which can effectively solve the "memory wall" problem.

[0060] By employing both runtime and offline strategies, PIMPR achieved a 5.78x performance improvement and a 6.05x energy reduction. Using both strategies increased throughput and resulted in a more balanced workload on top of the basic design.

[0061] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A personalized recommendation system based on in-memory processing, characterized in that, include: Bottom fully connected engine, embedded engine, buffer, top fully connected engine, and central processing unit; The input data for the bottom fully connected engine is the user feature vector; The input data of the embedding engine is a user interaction vector; the input end of the buffer is connected to the output end of the bottom fully connected engine and the output end of the embedding engine respectively, and the buffer is used to cache the output data of the bottom fully connected engine and the embedding engine. The input of the top fully connected engine is connected to the output of the buffer. The top fully connected engine is used to calculate the data cached in the buffer to obtain the recommendation result. The central processing unit is connected to the bottom fully connected engine, the embedded engine, the buffer, and the top fully connected engine respectively, and is used to control the operation of the bottom fully connected engine, the embedded engine, the buffer, and the top fully connected engine respectively; The embedding engine includes an embedding module and a prediction module; the embedding module consists of a ReRAM cross array, which is represented as an embedding array, and each row of the embedding array stores a user interaction vector; the prediction module consists of a set of vertically arranged ReRAM cross arrays, which is represented as a prediction array, and each column of the prediction array is connected to the corresponding row in the embedding array.

2. The personalized recommendation system based on in-memory processing according to claim 1, characterized in that, The embedding engine also includes a dual in-line memory module.

3. The personalized recommendation system based on in-memory processing according to claim 1, characterized in that, The embedding engine processes the user interaction vector in three phases: prediction, skipping, and execution.

4. The personalized recommendation system based on in-memory processing according to claim 3, characterized in that, The embedding module is used to execute the execution phase, and the prediction module is used to execute the prediction phase and the execution phase.

5. The personalized recommendation system based on in-memory processing according to claim 3, characterized in that, The bottom fully connected engine, the top fully connected engine, and the embedded engine are all equipped with special functional modules; The special function modules include: a shift and addition unit, a scalar arithmetic unit, and a logic unit.

6. The personalized recommendation system based on in-memory processing according to claim 5, characterized in that, The special function module in the embedded engine is also equipped with a non-zero detection unit, which is used to perform the jump phase.

7. The personalized recommendation system based on in-memory processing according to claim 3, characterized in that, The embedding engine incorporates a delayed matching pipeline mechanism when processing the user interaction vector; the delayed matching pipeline mechanism is used to reorder the prediction phase, the jump phase, and the execution phase.

8. The personalized recommendation system based on in-memory processing according to claim 3, characterized in that, An offline strategy is embedded when the embedding engine processes the user interaction vectors; the offline strategy reorders the user interaction vectors stored in the embedding array and the prediction array.

Images