An in-memory bayesian inference acceleration architecture, working method and probability mapping method

Abstract

The application discloses an in-memory Bayesian inference acceleration architecture, which comprises a core cross array, a row driver, a write / input buffer, a current mirror and a WTA sensing module. The application makes full use of the multi-level adjustable storage characteristics of the nonvolatile device in the core cross array to realize efficient in-memory Bayesian inference. A probability mapping method for the in-memory Bayesian inference acceleration architecture is provided, which effectively quantizes the floating-point probability value in the Bayesian inference algorithm model to the current parameter of the nonvolatile device, so that the Bayesian inference algorithm model can be efficiently deployed in the in-memory acceleration architecture based on the nonvolatile device design.

Description

Technical Field

[0001] This invention belongs to the field of storage and Bayesian inference technology, specifically relating to an in-memory Bayesian inference acceleration architecture, working method, and probability mapping method. Background Technology

[0002] Many machine learning models based on neural networks have important applications in fields such as computer graphics, biomedicine, and natural language processing. However, traditional neural network models often have high requirements for the quantity and quality of training data and struggle to provide high interpretability of their outputs. In contrast, machine learning algorithms based on Bayesian inference have lower requirements for training data and offer significant advantages in providing interpretable predictions and estimating uncertainty.

[0003] Most current in-memory computing (IMC) architectures are primarily designed for neural network workloads, achieving higher computing power and energy efficiency than traditional computing architectures through the use of non-volatile memory technology. However, they are difficult to directly apply to accelerating Bayesian inference operations. This is because the core operation of Bayesian inference, namely the product of prior probabilities and likelihood probabilities, differs significantly from the multiply-add (MAC) operation commonly found in neural networks, making it difficult to directly apply existing IMC designs to accelerate Bayesian inference.

[0004] Meanwhile, many existing hardware architectures for Bayesian inference employ separate probability storage units and inference operation circuits, which have the following shortcomings: 1) slow inference speed and high energy consumption; 2) poor scalability; 3) limited computing power, making it difficult to support large-scale inference operations.

[0005] To address the aforementioned issues and further improve the energy efficiency and scalability of Bayesian inference hardware, thereby providing more powerful computing capabilities for many Bayesian inference-based applications, this application proposes an in-memory Bayesian inference acceleration architecture, working method, and probability mapping method. Summary of the Invention

[0006] The purpose of this invention is to provide an in-memory Bayesian inference acceleration architecture, working method, and probability mapping method to solve the problems that existing neural network workload designs cannot be directly applied to Bayesian inference operations, as well as the problems that existing Bayesian inference hardware has, such as slow inference speed, high power consumption, poor scalability, and limited computing power.

[0007] The specific technical solution of the present invention is as follows:

[0008] The primary objective of this invention is to provide an in-memory Bayesian inference acceleration architecture, comprising a core cross array, a row driver, a write / input buffer, a current mirror, and a WTA sensing module.

[0009] Furthermore, the core cross array includes several three-port non-volatile devices forming a K-row × (M*N) column array. The gate of the three-port non-volatile device is connected to the bit line BL, the drain is connected to the word line WL, and the source is connected to the source line ScL. The three-port non-volatile devices in the same row share a word line WL and a source line ScL, and the three-port non-volatile devices in the same column share a bit line BL.

[0010] Furthermore, the three-port non-volatile device is a FeFET.

[0011] The second objective of this invention is to provide a method for operating the in-memory Bayesian inference acceleration architecture as described above, comprising:

[0012] Preparation phase: Before the architecture starts working, data is stored for each three-port non-volatile device in the core cross array according to the probability values ​​in the Bayesian inference model that needs to be accelerated;

[0013] Working phase: Based on the value of each feature in the input sample, each bit line BL is set to 0 or 1; each word line WL is set to 1; for any three-port non-volatile device, when its corresponding bit line BL is 1, the three-port non-volatile device will generate a current flowing into WL / ScL, the magnitude of which depends on the probability value stored therein; according to Kirchhoff's current law, the current in each row of WL / ScL reflects the cumulative posterior probability value of the activation unit in that row, that is, it can accurately reflect the posterior probability calculation result of the Bayesian inference of the current sample; the current mirror and WTA module further sense the output of each row of the core cross array to complete the final classification step of Bayesian inference.

[0014] Furthermore, the current mirror and WTA module further sense the output of each row of the core cross array to complete the final classification step of Bayesian inference. Specifically, the current mirror copies the current of each row's WL / ScL and inputs it into the WTA sensing module. The WTA sensing module compares the current values ​​of all rows and outputs the row with the maximum current value.

[0015] Furthermore, the process of storing data for each three-port non-volatile device in the core cross array based on the probability values ​​in the Bayesian inference model to be accelerated specifically includes: mapping the probability values ​​to the storage state of the non-volatile device, and then writing to each three-port non-volatile device through the word line WL, bit line BL, and source line ScL.

[0016] A third objective of this invention is to provide a probability mapping method (i.e., a method for mapping probability values ​​to non-volatile device memory states) suitable for in-memory Bayesian inference acceleration architectures as described above, comprising:

[0017] Truncation phase: Replace the values ​​of elements in the probability matrix that are less than 0.1 times the maximum value with 0.1 times the maximum value;

[0018] Logarithmic field transformation stage: Convert the value of each element in the probability matrix into its natural logarithm;

[0019] Column normalization stage: Add a constant to all element values ​​in each column of the probability matrix so that the maximum value of each column is 1;

[0020] Quantization mapping stage: The number and range of quantization intervals are determined based on the number of threshold voltages and the range of conduction current values ​​of non-volatile devices. The value of each element in the probability matrix is ​​replaced with the conduction current value corresponding to its quantization interval, thus corresponding to the storage state of the non-volatile device that generates the conduction current.

[0021] The beneficial effects of this invention are as follows: The in-memory Bayesian inference acceleration architecture of this invention can accelerate the posterior probability calculation and classification steps of the Bayesian inference model. This architecture fully utilizes the multi-level adjustable storage characteristics of non-volatile devices. By mapping the probability values ​​in the Bayesian inference model to the states of non-volatile devices and storing them in the non-volatile devices, the output value of the core cross array accurately corresponds to the posterior probability obtained by multiplying the prior probability and the likelihood probability, thereby realizing Bayesian inference. Simultaneously, a probability mapping method for the in-memory Bayesian inference architecture is proposed, which can effectively map the floating-point probability values ​​in the Bayesian inference model algorithm to the current parameters of the non-volatile devices. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the in-memory Bayesian inference acceleration architecture of the present invention;

[0023] Figure 2 This is an example diagram illustrating how the probability values ​​are mapped to the storage states of non-volatile devices according to the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solution of this application, the following description, in conjunction with the accompanying drawings, further illustrates an in-memory Bayesian inference acceleration architecture, its working method, and a probability mapping method.

[0025] like Figure 1As shown, an in-memory Bayesian inference acceleration architecture includes a core crossover array, row drivers, write / input buffers, a current mirror, and a WTA sensing module. The core crossover array comprises several three-port non-volatile devices (NTVs) forming a K-row × (M*N) column array. The gate of each NTV is connected to the bit line BL, the drain to the word line WL, and the source to the source line ScL. NTVs in the same row share a word line WL and a source line ScL, and NTVs in the same column share a bit line BL. The NTVs are FeFETs.

[0026] One working method for the in-memory Bayesian inference acceleration architecture described above includes the following:

[0027] Preparation phase: Before the above architecture starts working, data is stored for each three-port non-volatile device in the core cross array according to the probability values ​​in the Bayesian inference model that needs to be accelerated.

[0028] Working phase: Based on the value of each feature in the input sample, each bit line BL is set to 0 or 1; each word line WL is set to 1; for any three-port non-volatile device, when its corresponding bit line BL is 1, the three-port non-volatile device will generate a current flowing into WL / ScL, the magnitude of which depends on the probability value stored therein; according to Kirchhoff's current law, the current in each row of WL / ScL reflects the cumulative posterior probability value of the activation unit in that row, that is, it can accurately reflect the posterior probability calculation result of the Bayesian inference of the current sample; the current mirror and WTA module further sense the output of each row of the core cross array to complete the final classification step of Bayesian inference.

[0029] Specifically, the current mirror and WTA module further sense the output of each row of the core cross array to complete the final classification step of Bayesian inference. This includes: the current mirror copies the current of each row's WL / ScL and inputs it to the WTA sensing module; the WTA sensing module compares the current values ​​of all rows and outputs the row with the highest current value. Based on the probability values ​​in the Bayesian inference model to be accelerated, data is stored for each three-port non-volatile device in the core cross array. This includes: mapping the probability values ​​to the non-volatile device's storage state, and then writing to each three-port non-volatile device through the word line WL, bit line BL, and source line ScL.

[0030] like Figure 2 As shown, the present invention also provides a method for mapping probability values ​​to non-volatile device storage states applicable to in-memory Bayesian inference acceleration architectures as described above, comprising the following:

[0031] Truncation phase: Replace the values ​​of elements in the probability matrix that are less than 0.1 times the maximum value with 0.1 times the maximum value;

[0032] Logarithmic field transformation stage: Convert the value of each element in the probability matrix into its natural logarithm;

[0033] Column normalization stage: Add a constant to all element values ​​in each column of the probability matrix so that the maximum value of each column is 1;

[0034] Quantization mapping stage: The number and range of quantization intervals are determined based on the number of threshold voltages and the range of conduction current values ​​of non-volatile devices. The value of each element in the probability matrix is ​​replaced with the conduction current value corresponding to its quantization interval, thus corresponding to the storage state of the non-volatile device that generates the conduction current.

[0035] Example

[0036] Continue reading Figure 1 It can be seen that the core cross array of an in-memory Bayesian inference acceleration architecture has a size of K×(M*N), and each unit consists of a non-volatile device that stores an element p of the probability matrix P. k，mn Each column of gates is connected to a common BL, corresponding to the feature value of the input sample. The drains in each row are connected to a single WL, corresponding to the posterior probability of the output.

[0037] During operation, each active device generates a corresponding current i. k，mn Current i on all rows k Together they constitute the posterior probability Right now The inference results applied to the Bayesian model corresponding to P.

[0038] Continue reading Figure 2 It can be seen that, Figure 2 An example of a probabilistic mapping method for accelerating in-memory Bayesian inference architectures is shown.

[0039] As can be seen, each element in the original probability matrix P takes values ​​ranging from 0 to 1.

[0040] 1) During the truncation phase, the value range of each element in P is updated to 0.1 to 1 by truncating and replacing the value of each element in P;

[0041] 2) During the logarithmic domain transformation stage, by taking the logarithm of each element in the probability matrix P, the range of values ​​for the elements in P is updated to -2.3 to 0;

[0042] 3) In the column normalization stage, the probability matrix P is normalized by column normalization, and the range of values ​​of the elements in P is updated to -1.3 to 1;

[0043] 4) In the quantization mapping stage, based on the fact that the number of threshold voltages of non-volatile devices is 4 and the conduction current value range is 0.1-1.0μA, the number of quantization intervals is determined to be 4, and the probability quantization ranges are -1.3 to -0.725, -0.725 to -0.15, -0.15 to 0.425, and 0.425 to 1, respectively. The conduction current values ​​corresponding to each quantization range are 0.1μA, 0.4μA, 0.7μA, and 1μA, respectively. Each element in the probability matrix P is replaced with the corresponding conduction current value according to the quantization interval range in which its value is located. The value range of the elements in P is updated to {0.1μA, 0.4μA, 0.7μA, 1μA}.

[0044] Through the above series of operations, the value of each element in the probability matrix is ​​mapped to a quantized on-current value, and each quantized on-current value naturally corresponds to a storage state of a non-volatile device. Therefore, this method can map probability values ​​to storage states of non-volatile devices.

[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A working method for an in-memory Bayesian inference acceleration architecture, characterized in that, The in-memory Bayesian inference acceleration architecture includes a core crossover array, row drivers, write / input buffers, a current mirror, and a WTA sensing module. The core crossover array includes several three-port non-volatile devices forming a K-row × (M*N) column array. The gate of each three-port non-volatile device is connected to the bit line BL, the drain is connected to the word line WL, and the source is connected to the source line ScL. Three-port non-volatile devices in the same row share a word line WL and a source line ScL, and three-port non-volatile devices in the same column share a bit line BL. The working method includes: Preparation phase: Before the architecture starts working, data is stored for each three-port non-volatile device in the core cross array according to the probability values ​​in the Bayesian inference model that needs to be accelerated; Working phase: Based on the value of each feature in the input sample, each bit line BL is set to 0 or 1; each word line WL is set to 1; for any three-port non-volatile device, when its corresponding bit line BL is 1, the three-port non-volatile device will generate a current flowing into WL / ScL, the magnitude of which depends on the probability value stored therein; according to Kirchhoff's current law, the current in each row of WL / ScL reflects the cumulative posterior probability value of the activation unit in that row, that is, it can accurately reflect the posterior probability calculation result of the Bayesian inference of the current sample; the current mirror and WTA module further sense the output of each row of the core cross array to complete the final classification step of Bayesian inference.

2. The working method of the in-memory Bayesian inference acceleration architecture according to claim 1, characterized in that, The three-port non-volatile device is a FeFET.

3. The working method of the in-memory Bayesian inference acceleration architecture according to claim 1, characterized in that, The current mirror and WTA module further sense the output of each row of the core cross array to complete the final classification step of Bayesian inference. Specifically, the current mirror copies the current of each row's WL / ScL and inputs it into the WTA sensing module. The WTA sensing module compares the current values ​​of all rows and outputs the row with the maximum current value.

4. The working method of the in-memory Bayesian inference acceleration architecture according to claim 1, characterized in that, The process of storing data for each three-port non-volatile device in the core cross array based on the probability values ​​in the Bayesian inference model to be accelerated specifically includes: mapping the probability values ​​to the storage state of the non-volatile device, and then writing to each three-port non-volatile device through the word line WL, bit line BL, and source line ScL.

5. A probability mapping method for an in-memory Bayesian inference acceleration architecture, applicable to the in-memory Bayesian inference acceleration architecture of claim 1, characterized in that, The mapping method includes: Truncation phase: Replace the values ​​of elements in the probability matrix that are less than 0.1 times the maximum value with 0.1 times the maximum value; Logarithmic field transformation stage: Convert the value of each element in the probability matrix into its natural logarithm; Column normalization stage: Add a constant to all element values ​​in each column of the probability matrix so that the maximum value of each column is 1; Quantization mapping stage: The number and range of quantization intervals are determined based on the number of threshold voltages and the range of conduction current values ​​of non-volatile devices. The value of each element in the probability matrix is ​​replaced with the conduction current value corresponding to its quantization interval, thus corresponding to the storage state of the non-volatile device that generates the conduction current.

Images