A forward gradient regression accelerator based on non-volatile memory and operation method

Abstract

The present invention discloses a forward gradient regression accelerator based on non-volatile memory and an operation method. The forward gradient regression accelerator includes a control module, an in-memory processing module and an operation module. The control module includes a DRAM memory for storing operation data and transmitting corresponding instructions and data to the in-memory processing module and the operation module. The in-memory processing module includes a non-volatile memory array for calculating the cosine similarity between the residual vector and the regression variable for forward gradient regression and transmitting the operation result to the control module. The operation module is used to receive the instructions and data from the control module, complete the data mapping operation and update the regression coefficient and the residual vector. The present invention uses a non-volatile memory array to perform large-scale matrix-vector multiplication operations in a 0(1) manner, and cooperates with the DRAM memory to perform a numerical iterative algorithm with backup data. While achieving accelerated solution to the forward gradient regression problem, it can ensure calculation accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of analog circuits, and more particularly, relates to a forward gradient regression accelerator based on a non-volatile memory and an operating method thereof. Background Art

[0002] Forward stagewise regression (FGR) is an algorithm widely used in image and signal processing. It can be used for a variety of data-intensive tasks, such as feature selection, sparse coding, matching pursuit, and sparse estimation. FGR can enhance the predictive accuracy and interpretability of statistical models, achieving similar results to lasso regression. However, FGR typically involves a large number of matrix-vector multiplication operations, which severely limits its efficiency.

[0003] The in-memory computing architecture based on non-volatile memory is an emerging architecture for processing data-intensive tasks. It can accelerate operations while reducing power consumption. This architecture chooses to perform operations directly in the memory, which can reduce data transmission during the operation and thus reduce power consumption. At the same time, this architecture uses analog circuits to perform highly parallel operations, which can accelerate the calculation of matrix-vector multiplication in a 0(1) manner. Currently, the in-memory computing architecture has received widespread attention in the field of microelectronics and has great potential in improving the energy efficiency of various machine learning algorithms.

[0004] However, in-memory computing architectures based on non-volatile memory still face many challenges in improving the energy efficiency of machine learning algorithms. For example, hardware limitations limit the number of memory states and the accuracy of analog-to-digital converters (ADCs) or digital-to-analog converters (DACs), while data precision is typically higher. Furthermore, errors in the converters or memory can affect the precision of calculations, reducing the stability and accuracy of the results.

[0005] Therefore, how to use in-memory computing to improve the energy efficiency of forward gradient regression while ensuring computational accuracy is an urgent problem that needs to be solved. Summary of the Invention

[0006] In view of the shortcomings of the prior art, the present invention aims to provide a forward gradient regression accelerator based on non-volatile memory and an operating method, which can improve the energy efficiency of solving the forward gradient regression problem while ensuring its computational accuracy.

[0007] To achieve the above objectives, in a first aspect, the present invention provides a forward gradient regression accelerator based on non-volatile memory, comprising a control module, an in-memory processing module and a calculation module, wherein:

[0008] A control module includes a DRAM memory and a first control unit, the DRAM memory being used to store data required for solving the forward gradient regression problem and data returned by the in-memory processing module and the operation module, the first control unit being used to control the DRAM memory to transmit corresponding data and instructions to the in-memory processing module and the operation module according to the forward gradient regression problem to be solved, the data including the variable matrix, the residual vector and the regression coefficient, the instructions including the data mapping instruction and the update instruction, the update instruction being transmitted each time the DRAM memory receives data returned by the in-memory processing module;

[0009] an operation module, comprising a logic operation unit and a third control unit, the third control unit being configured to control the logic operation unit to complete data mapping of the variable matrix and the residual vector according to the data mapping instruction, the variable matrix, and the residual vector, and to determine the variable most relevant to the residual vector according to the update instruction, control the logic operation unit to complete numerical updates of the residual vector and the regression coefficient according to the variable, and transmit the updated numerical values ​​to the DRAM memory;

[0010] The in-memory processing module includes a second control unit and a matrix operation unit. The matrix operation unit includes a non-volatile memory array, a DAC converter, a transimpedance amplifier, and an ADC converter. The second control unit is used to encode each element of the variable matrix after data mapping as the conductance difference between two non-volatile memories in the non-volatile memory array; the DAC converter is used to convert the residual vector value after each update into a voltage and input it into the non-volatile memory array, and enable the non-volatile memory array to complete the multiplication and addition operation between the residual vector and the regression variable in the variable matrix. The transimpedance amplifier and ADC converter are used to perform analog-to-digital conversion on the operation result data and send it to the DRAM memory.

[0011] In one embodiment, the non-volatile memory includes a resistive memory, a phase change memory, a NOR FLASH memory, a spin transfer torque memory, or a ferroelectric field effect transistor.

[0012] In one embodiment, the logic operation unit includes an addition and subtraction operation circuit and a multiplication and division operation circuit.

[0013] In one embodiment, the forward gradient regression problem to be solved includes feature selection, sparse coding, matching pursuit or sparse estimation problem.

[0014] In a second aspect, the present invention provides an operating method of the non-volatile memory-based forward gradient regression accelerator, comprising the following steps:

[0015] (1) When it is necessary to solve the forward gradient regression problem to be solved, the first control unit selects a target instruction set from a pre-stored algorithm instruction set according to the forward gradient regression problem to be solved, and transmits the data mapping instruction and the variable matrix in the target instruction set from the DRAM memory to the third control unit. The third control unit completes the data mapping processing of the variable matrix through the control logic operation unit, and then transmits the variable matrix after the data mapping processing to the second control unit through the first control unit. Finally, the second control unit encodes each element of the initialized variable matrix as the conductance difference between the two non-volatile memories in the non-volatile memory array;

[0016] (2) The first control unit transmits the data mapping instruction and the residual vector in the target instruction set from the DRAM memory to the third control unit. The third control unit completes the data mapping of the residual vector through the logic control operation unit, and then transmits the residual vector after data mapping to the in-memory processing module through the first control unit. The DAC converter in the in-memory processing module converts it into a voltage and inputs it into the non-volatile memory array. Then, the non-volatile memory array completes the multiplication and addition operation between the residual vector and the regression variable in the variable matrix. The transimpedance amplifier and the ADC converter convert the operation result into a digital signal, and the second control unit sends the operation result to the first control unit, which transmits it to the DRAM memory for storage.

[0017] (3) After receiving the calculation result, the first control unit sends an update instruction and the calculation result to the calculation module. The calculation module determines the maximum absolute value in the calculation result. The variable corresponding to the maximum value is the variable most correlated with the residual vector. The calculation module then uses the variable with a step size to update the residual vector and the regression coefficient. Finally, the third control unit sends the updated residual vector and regression coefficient to the first control unit, which transfers them to the DRAM memory for storage.

[0018] (4) The first control unit determines whether the number of iterations reaches the upper limit or whether the L2 norm of the residual vector is lower than the threshold. If not, repeat the above steps (2) to (4) until the first control unit determines that the number of iterations reaches the upper limit or the L2 norm of the residual vector is lower than the threshold, and the solution of the forward gradient regression problem to be solved is completed.

[0019] In one embodiment, the residual vector initialization mapping data is an input signal, and the regression coefficient is initialized to 0.

[0020] In one embodiment, in step (2), the initialization mapping method of the residual vector is:

[0021] Calculate the average of the absolute values ​​of all vector elements in the residual vector;

[0022] Points at equal intervals around 0 are mapped to the corresponding vector elements, and the intervals are set according to the average value.

[0023] In one embodiment, in step (2), the third control unit completes data mapping of the residual vector according to the type of forward gradient regression problem to be solved; when solving the feature selection problem, the residual vector is the difference between all elements in the input signal and the reconstructed signal; when solving the image restoration problem based on sparse coding, the residual vector is the difference between all non-zero elements in the input signal and the reconstructed signal.

[0024] In one embodiment, in step (3), the operation module uses the variable with a step size to update the residual vector and the regression coefficient, specifically:

[0025] Update the residual vector according to the positive or negative result of the multiplication and addition operation of the residual vector and the variable. When it is positive, subtract the variable of one step length from the residual vector. When it is negative, add the variable of one step length to the residual vector.

[0026] The regression coefficient is updated according to the positive or negative result of the multiplication and addition operation of the residual vector and the variable. When it is positive, the regression coefficient is increased by a step size, and when it is negative, the regression coefficient is decreased by a step size.

[0027] In one embodiment, in step (4), after every two or more iterations, the first control unit determines whether the L2 norm of the residual vector is lower than a threshold.

[0028] The present invention provides a non-volatile memory-based forward gradient regression accelerator and an operating method thereof. The accelerator employs mixed digital-analog operations, i.e., employs non-volatile memory-based analog operations to accelerate the calculation of cosine distance for forward gradient regression, searches for the variable most correlated with the residual vector, and uses digital operations and full-precision data backed up in a DRAM memory to complete numerical iteration. This prevents data and operation results from being degraded by the non-ideal characteristics of the non-volatile memory, enabling the accelerator provided by this embodiment to accelerate the solution of the forward gradient regression problem while achieving good computational accuracy. Furthermore, because the accelerator provided by this embodiment employs in-memory computation to accelerate the solution of the forward gradient regression problem, and because forward gradient regression has a wide range of applications, the accelerator provided by this embodiment can be used to accelerate the solution of various sparse model problems, such as feature selection, sparse coding, matching pursuit, and sparse estimation, and thus has high reusability. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a block diagram of a non-volatile memory-based forward gradient regression accelerator according to an embodiment of the present invention;

[0030] Figure 2 is a flowchart of an operating method of a non-volatile memory-based forward gradient regression accelerator provided by one embodiment of the present invention;

[0031] Figure 3 This is a circuit schematic diagram of a matrix operation unit in an in-memory processing module provided by one embodiment of the present invention;

[0032] Figure 4 yes Figure 3 The provided operation method implements the data flow diagram of sparse coding. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0034] In order to improve the energy efficiency of the traditional in-memory computing architecture in calculating the forward gradient regression problem and ensure its calculation accuracy, the present invention provides a forward gradient regression accelerator based on non-volatile memory, including a control module 10, an in-memory processing module 20 and a calculation module 30.

[0035] It should be noted that solving the forward gradient regression problem mainly involves two parts: one is calculating the cosine distance between the residual vector and the regression variables in the variable matrix, and the other is iterating the regression coefficients and residual vector. The result of this calculation is the selection of some variables to represent the input signal. Based on this, the forward gradient regression accelerator provided by the present invention uses the in-memory processing module 20 to calculate the cosine distance and the calculation module 30 to iterate the regression coefficients and residual vector.

[0036] like Figure 1 As shown, the control module 10 provided in this embodiment includes a DRAM memory 12 and a first control unit 11. The in-memory processing module 20 includes a matrix operation unit 22 and a second control unit 21. The matrix operation unit 22 includes a non-volatile memory array, a DAC converter, a transimpedance amplifier, and an ADC converter. The operation module 30 includes a logic operation unit 32 and a third control unit 31.

[0037] In this embodiment, the control module 10 is used to coordinate the in-memory processing module 20 and the computation module 30 to jointly solve the forward gradient regression problem. Specifically, the DRAM memory 12 in the control module 10 is used to store the data required to solve the forward gradient regression problem and the data returned by the in-memory processing module and the computation module. The first control unit 11 is used to control the DRAM memory 12 to transmit corresponding data and instructions to the in-memory processing module 20 and the computation module 30 based on the forward gradient regression problem to be solved, thereby controlling the entire accelerator. The data includes variable matrices, residual vectors, and regression coefficients, and the instructions include data mapping instructions and update instructions. The update instructions are transmitted by the DRAM memory each time the DRAM memory receives data returned by the in-memory processing module.

[0038] The operation module 30 is used to receive the corresponding instructions and data of the control module 10, complete the operation of the residual vector (the difference between the input signal and the reconstructed signal) and the variable matrix data mapping, and is used to update the regression coefficient and the residual vector. Its specific implementation method is as follows: the third control unit 31 in the operation module 30 is used to control the logic operation unit according to the data mapping instruction, the variable matrix, and the residual vector to complete the data mapping of the variable matrix and the residual vector, and is also used to determine the variable most relevant to the residual vector according to the update instruction, and control the logic operation unit according to the variable to complete the numerical update of the residual vector and the regression coefficient, and transmit the updated numerical value to the first control unit, which is transmitted to the DRAM memory for storage. Specifically, the logic operation unit provided in this embodiment adopts an addition and subtraction operation circuit and a multiplication and division operation circuit.

[0039] The in-memory processing module 20 is used to calculate the cosine similarity between the residual vector and the variable matrix for forward gradient regression, and transmit the calculation results to the DRAM memory 12 in the control module 10 for storage. Its specific implementation method is as follows: the second control unit 21 is used to encode each element in the variable matrix after data mapping as the conductance difference between two non-volatile memories in the non-volatile memory array; the DAC converter is used to convert the residual vector value after each update into a voltage and input it into the non-volatile memory array, and the non-volatile memory array is used to complete the multiplication and addition operation between the residual vector and the regression variables in the variable matrix. The transimpedance amplifier and ADC converter are used to perform analog-to-digital conversion on the calculation result data and send it to the first control unit, which transmits it to the DRAM memory for storage.

[0040] In the in-memory processing module 20, the operation form of the matrix operation unit 22 is a multiplication and addition operation, that is, multiple voltage groups V1, V2, ..., V n Input and multiple groups of nonvolatile memory conductance G1, G2, ..., G nThe corresponding multiplication is performed respectively, and then the products are added together, such as: V1×G1+V2×G2+...,+V n ×G n The final output operation result will be converted into a digital signal by the transimpedance amplifier and the ADC converter and transmitted to the DRAM memory 12 in the control module for storage. Preferably, the non-volatile memory in the non-volatile memory array provided by the present invention can adopt non-volatile memories such as resistive random access memory (RRAM), phase change memory (PCM), NORFLASH memory, spin transfer torque memory (STT-MRAM) or ferroelectric field effect transistor (FeFET), and this embodiment does not limit it.

[0041] The non-volatile memory-based forward gradient regression accelerator provided in this embodiment uses mixed digital-analog operations, that is, analog operations based on non-volatile memory are used to accelerate the calculation of cosine distance for forward gradient regression, find the variable most correlated with the residual vector, and use digital operations and full-precision data backed up in DRAM memory to complete numerical iteration. This can prevent data and calculation results from being degraded by the non-ideal characteristics of non-volatile memory, so that the accelerator provided by this embodiment has good computational accuracy while accelerating the solution of the forward gradient regression problem. At the same time, because the accelerator provided by this embodiment uses in-memory computing to accelerate the solution of the forward gradient regression problem, and because forward gradient regression has a wide range of applications, the accelerator provided by this embodiment can be used to accelerate the solution of various sparse model problems such as feature selection, sparse coding, matching pursuit, and sparse estimation, and has high reusability.

[0042] Figure 2 is a flowchart of an operation method of the non-volatile memory-based forward gradient regression accelerator provided by an embodiment of the present invention. In the flowchart, β1, β2, ..., β m Represents the regression coefficient in the regression equation, X represents the variable matrix, y represents the input signal, Represents the reconstructed signal, ε represents the step size, k represents the number of iterations, represents the residual vector.

[0043] The operation method provided in this embodiment is as follows:

[0044] S10, when it is necessary to solve the forward gradient regression problem to be solved, the first control unit selects a target instruction set from a pre-stored algorithm instruction set according to the forward gradient regression problem to be solved, and transmits the data mapping instructions and variable matrix in the target instruction set from the DRAM memory to the third control unit. The third control unit completes the data mapping processing of the variable matrix through the control logic operation unit, and then transmits the variable matrix after the data mapping processing to the second control unit through the first control unit. Finally, the second control unit encodes each element of the initialized variable matrix as the conductance difference between the two non-volatile memories in the non-volatile memory array.

[0045] Preferably, the regression coefficient β is initially 0.

[0046] Preferably, the mapping method of the variable matrix X can be: (1) first standardize each variable (z-score standardization), (2) map all values ​​according to the designed accuracy and mapping range. Then use the difference in conductance between the two non-volatile memories (G + -G - ) to encode the mapped value.

[0047] S20, the first control unit maps the data in the target instruction set to the instruction and the residual vector The residual vector is transmitted from the DRAM memory to the third control unit, and the third control unit completes the residual vector through the logic control operation unit. The residual vector after data mapping is then transmitted to the in-memory processing module through the first control unit, and the DAC converter in the in-memory processing module converts it into a voltage and inputs it into the non-volatile memory array. The non-volatile memory array then completes the residual vector. The multiplication and addition operation between the regression variables in the variable matrix X is then performed, and the operation result is converted into a digital signal by the transimpedance amplifier and the ADC converter, and sent by the second control unit to the DRAM memory in the control module for storage.

[0048] In step S20, when the third control unit performs data mapping on the residual vector, the mapping method may be changed according to the type of forward gradient regression problem to be solved. For example, when solving a feature selection problem, the residual vector is the difference between all elements in the input signal and the reconstructed signal; when solving a sparse coding-based image restoration problem, the residual vector is the difference between all non-zero elements in the input signal and the reconstructed signal.

[0049] In step S20, the multiplication and addition operation performed by the non-volatile memory array is to calculate the residual vector according to Ohm's law I=V×G and Kirchhoff's current law. and the cosine distance between the regressors in the variable matrix X. Specifically, Figure 3 As shown, the output of each column of the nonvolatile memory array is Thus, each column of current represents a regression variable and a residual vector The cosine distance between them is then converted into a voltage signal by a transimpedance amplifier (TIA) and then converted into a digital signal by an ADC converter.

[0050] Preferably, the reconstructed signal Initialized to 0, the residual vector Initialized to the input signal.

[0051] Preferably, the residual vector The mapping method is as follows: (1) calculate the average value of the absolute values ​​of all vector elements in the residual vector, (2) take some points at equal intervals around 0 to map all nearby values, the interval depends on the previously calculated average value. Then use the DAC converter to convert the mapped values ​​into voltages V1, V2, ..., V n and input into the non-volatile memory array.

[0052] S30, the first control unit sends an update instruction and the calculation result data to the calculation module after receiving the calculation result. The calculation module determines the maximum absolute value of the calculation result. The variable corresponding to the maximum value is the variable j that is most correlated with the residual vector. The calculation module then uses this variable with a step size to update the reconstructed signal. and the residual vector And update the regression coefficient β j Finally, the first control unit stores the updated residual vector and regression coefficient in the DRAM memory.

[0053] In step S30, the variable data used to update the residual vector is full-precision data stored in the DRAM memory.

[0054] Preferably, the reconstructed signal may not be updated, but the residual vector may be updated directly.

[0055] Preferably, the operation module uses the variable with a step size of ε to update the residual vector, and the method of updating the regression coefficient is specifically as follows: updating the residual vector according to the positive or negative result of the multiplication and addition operation of the residual vector and the variable, when it is positive, subtracting the variable with a step size of ε from the residual vector, and when it is negative, adding the variable with a step size of ε to the residual vector; updating the regression coefficient β according to the positive or negative result of the multiplication and addition operation of the residual vector and the variable j , when it is positive, let the regression coefficient β j Add a step size ε, when it is negative, let the regression coefficient β j Subtract a step size ε.

[0056] S40, the first control unit determines whether the number of iterations reaches the upper limit or whether the L2 norm of the residual vector is lower than the threshold. If not, repeat the above steps S20 to S40 until the first control unit determines that the number of iterations reaches the upper limit or the L2 norm of the residual vector is lower than the threshold, thereby completing the solution of the forward gradient regression problem to be solved.

[0057] In step S40 , the conductance values ​​stored in the non-volatile memory array do not change during the numerical iterations, and a single write operation is required to execute the numerical iteration loop until the loop terminates. In particular, when solving sparse coding problems, a single write operation (saving the dictionary to the non-volatile memory array) is required to perform sparse coding on any image, eliminating the need for further write operations and effectively reducing data transmission.

[0058] To reduce computational overhead, step S40 may be configured to determine whether the L2 norm of the residual vector is below a threshold after two or more iterations. To further reduce computational overhead, when performing data mapping on the residual vector, the average of the absolute values ​​of the vector elements may be calculated after two or more iterations.

[0059] To more clearly illustrate the operation method of the non-volatile memory-based forward gradient regression accelerator provided by this embodiment, the operation method is used to perform a sparse coding task for detailed description below.

[0060] It should be noted that when performing sparse coding tasks, the variable matrix of forward gradient regression is the sparse coding dictionary, the residual vector is the difference between the original image and the reconstructed image, the variable j is the dictionary element j most correlated with the residual vector, and the result of the operation is to select some elements in the dictionary to represent the input digital image signal.

[0061] like Figure 4 As shown, when the forward gradient regression accelerator provided by the present invention is used to perform sparse coding tasks, the operation method is as follows:

[0062] Step 1: When the forward gradient regression problem to be solved needs to be solved, the first control unit selects a target instruction set from the pre-stored algorithm instruction set according to the forward gradient regression problem to be solved. Then, the first control unit transmits the instruction and the data of the dictionary matrix X to the third control unit through the DRAM memory, and the operation module performs data mapping processing on the regression coefficient β and the dictionary matrix X. Then, the dictionary matrix after data mapping processing is transmitted to the second control unit through the first control unit, and finally the second control unit encodes each element of the dictionary matrix as the conductance difference (G + -G - ).

[0063] Preferably, the regression coefficient β is initialized to 0 and the image is reconstructed Initialized to 0, the residual vector Initialized to the input signal y.

[0064] Preferably, the data mapping process of the dictionary matrix X involves first performing z-score normalization on each dictionary element and then mapping each dictionary element to 15 integers in the interval [-7, 7] according to a certain ratio. In this embodiment, the mapping of dictionary elements is not limited to the interval [-7, 7]. The specific interval range and number of integers are determined by the number of states of the non-volatile memory.

[0065] Step 2: The first control unit converts the partial instruction and residual vector The data is transferred from the DRAM memory to the third control unit, and the residual vector is completed by the calculation module The data is mapped and then transmitted to the in-memory processing module, where the DAC converter converts the value into voltage. Then, the in-memory processing module calculates the residual vector and the cosine distance between each dictionary element, and transmits the operation result to the control module through the second control unit.

[0066] Preferably, the data mapping of the residual vector is as follows: (1) the average value of the absolute values ​​of the vector elements is calculated; (2) 15 points are taken at equal intervals around 0 to map all nearby values, the interval being determined by the previously calculated average value; and (3) the mapped values ​​are converted into voltages V1, V2, ..., V using a DAC converter. n and input into the non-volatile memory array.

[0067] Preferably, the number of points selected near 0 is determined by the accuracy of the DAC, and this embodiment is not limited to selecting only 15 points.

[0068] Step 3: The first control unit transmits the received operation result to the operation module, which determines the maximum absolute value of the operation result. The maximum value corresponds to the most relevant dictionary element j. Then, the operation module uses the dictionary element with a step size to update the reconstructed image. and the residual vector And update the regression coefficient β j Finally, the first control unit stores the residual vector and regression coefficient in the DRAM memory.

[0069] Step 4: The first control unit determines whether the number of iterations k reaches the upper limit or the L2 norm of the residual vector Is it lower than the threshold and determine whether to terminate the iteration loop.

[0070] Step 5: Repeat steps 2 to 4 until the first control unit determines that the number of iterations k reaches the upper limit or the L2 norm of the residual vector If the value is lower than the threshold, the problem is solved.

[0071] It should be noted that Figure 4 The external processor in the system consists of a control module and an operation module.

[0072] Furthermore, when this accelerator is used to perform sparse coding tasks to repair an image with some missing pixels, the residual vector is the difference between the remaining pixels in the image and the reconstructed pixels. When performing this task, the accelerator's operation remains essentially unchanged; only the calculation of the residual vector is modified. Beyond the numerical problems described above, this accelerator can also be applied to any problem that can be solved by forward gradient regression, and this embodiment does not impose any limitations.

[0073] In summary, the present invention has the following advantages: (1) The accelerator provides a method for accelerating the solution of forward gradient regression using in-memory calculations. Since forward gradient regression has a wide range of applications, the accelerator can be used to accelerate the solution of various sparse model problems such as feature selection, sparse coding, matching pursuit and sparse estimation, and has high reusability. (2) The accelerator fully utilizes the advantages of mixed digital and analog operations. Analog operations based on non-volatile memory are used to accelerate the calculation of cosine distance for forward gradient regression and find the variables most correlated with the residual vector. Digital operations and full-precision data backed up in DRAM are used to complete numerical iterations. Therefore, the accelerator can achieve good computational accuracy while accelerating the solution of forward gradient regression. (3) When mapping high-precision data to low-precision voltage and conductance, the mapping interval is set according to the average value, so that the low-precision voltage and conductance reflect the numerical distribution of the original data as much as possible. This mapping method can avoid outliers from excessively affecting the mapping interval, and to a certain extent overcome the adverse effects brought about by limited hardware precision.

[0074] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A forward gradient regression accelerator based on non-volatile memory, characterized in that: It includes a control module, an in-memory processing module and an operation module, wherein: A control module includes a DRAM memory and a first control unit, the DRAM memory being used to store data required for solving a forward gradient regression problem and data returned by the in-memory processing module and the operation module, the first control unit being used to control the DRAM memory to transmit corresponding data and instructions to the in-memory processing module and the operation module according to the forward gradient regression problem to be solved, the data including a variable matrix, a residual vector, and a regression coefficient, the instructions including a data mapping instruction and an update instruction, the update instruction being transmitted each time the DRAM memory receives data returned by the in-memory processing module; an operation module, comprising a logic operation unit and a third control unit, the third control unit being configured to control the logic operation unit to complete data mapping of the variable matrix and the residual vector according to the data mapping instruction, the variable matrix, and the residual vector, and to determine the variable most relevant to the residual vector according to the update instruction, control the logic operation unit to complete numerical updates of the residual vector and the regression coefficient according to the variable, and transmit the updated numerical values ​​to the DRAM memory; The in-memory processing module includes a second control unit and a matrix operation unit. The matrix operation unit includes a non-volatile memory array, a DAC converter, a transimpedance amplifier, and an ADC converter. The second control unit is used to encode each element of the variable matrix after data mapping as the conductance difference between two non-volatile memories in the non-volatile memory array; the DAC converter is used to convert the residual vector value after each update into a voltage and input it into the non-volatile memory array, and enable the non-volatile memory array to complete the multiplication and addition operation between the residual vector and the regression variable in the variable matrix. The transimpedance amplifier and ADC converter are used to perform analog-to-digital conversion on the operation result data and send it to the DRAM memory.

2. The non-volatile memory-based forward gradient regression accelerator according to claim 1, characterized in that: Non-volatile memory includes resistive memory, phase change memory, NOR FLASH memory, spin transfer torque memory or ferroelectric field effect transistor.

3. The non-volatile memory-based forward gradient regression accelerator according to claim 1, wherein: The logic operation unit includes an addition and subtraction operation circuit and a multiplication and division operation circuit.

4. The non-volatile memory-based forward gradient regression accelerator according to claim 1, wherein: The forward gradient regression problem to be solved includes feature selection, sparse coding, matching pursuit or sparse estimation problems.

5. A method for operating a non-volatile memory-based forward gradient regression accelerator according to any one of claims 1 to 4, characterized in that: The steps include: (1) When it is necessary to solve the forward gradient regression problem to be solved, the first control unit selects a target instruction set from a pre-stored algorithm instruction set according to the forward gradient regression problem to be solved, and transmits the data mapping instruction and the variable matrix in the target instruction set from the DRAM memory to the third control unit. The third control unit completes the data mapping processing of the variable matrix through the control logic operation unit, and then transmits the variable matrix after the data mapping processing to the second control unit through the first control unit. Finally, the second control unit encodes each element of the variable matrix after the data mapping processing as the conductance difference between the two non-volatile memories in the non-volatile memory array; (2) The first control unit transmits the data mapping instruction and the residual vector in the target instruction set from the DRAM memory to the third control unit. The third control unit completes the data mapping of the residual vector through the logic control operation unit, and then transmits the residual vector after data mapping to the in-memory processing module through the first control unit. The DAC converter in the in-memory processing module converts it into a voltage and inputs it into the non-volatile memory array. The non-volatile memory array then completes the multiplication and addition operation between the residual vector and the regression variable in the variable matrix. The transimpedance amplifier and the ADC converter then convert the operation result into a digital signal. The second control unit sends the operation result to the first control unit, and the first control unit transmits it to the DRAM memory for storage. (3) After receiving the calculation result, the first control unit sends an update instruction and the calculation result to the calculation module. The calculation module determines the maximum absolute value in the calculation result. The variable corresponding to the maximum value is the variable most correlated with the residual vector. The calculation module then uses the variable with a step size to update the residual vector and the regression coefficient. Finally, the third control unit sends the updated residual vector and regression coefficient to the first control unit, which transfers them to the DRAM memory for storage. (4) The first control unit determines whether the number of iterations reaches the upper limit or whether the L2 norm of the residual vector is lower than the threshold. If not, repeat the above steps (2) to (4) until the first control unit determines that the number of iterations reaches the upper limit or the L2 norm of the residual vector is lower than the threshold, and the solution of the forward gradient regression problem to be solved is completed.

6. The method for operating a forward gradient regression accelerator based on non-volatile memory according to claim 5, wherein: The residual vector initialization mapping data is an input signal, and the regression coefficient is initialized to 0.

7. The method for operating a forward gradient regression accelerator based on non-volatile memory according to claim 5, wherein: In step (2), the initialization mapping method of the residual vector is: Calculate the average of the absolute values ​​of all vector elements in the residual vector; Points at equal intervals around 0 are mapped to the corresponding vector elements, and the intervals are set according to the average value.

8. The method for operating a forward gradient regression accelerator based on non-volatile memory according to claim 5, wherein: In step (2), the third control unit completes the data mapping of the residual vector according to the type of the forward gradient regression problem to be solved; when solving the feature selection problem, the residual vector is the difference between all elements in the input signal and the reconstructed signal; when solving the image restoration problem based on sparse coding, the residual vector is the difference between all non-zero elements in the input signal and the reconstructed signal.

9. The method for operating a forward gradient regression accelerator based on non-volatile memory according to claim 5, wherein: In step (3), the operation module uses the variable with a step size to update the residual vector and the regression coefficient, specifically: Update the residual vector according to the positive or negative result of the multiplication and addition operation of the residual vector and the variable. When it is positive, subtract the variable of one step length from the residual vector. When it is negative, add the variable of one step length to the residual vector. The regression coefficient is updated according to the positive or negative result of the multiplication and addition operation of the residual vector and the variable. When it is positive, the regression coefficient is increased by a step size, and when it is negative, the regression coefficient is decreased by a step size.

10. The method for operating a forward gradient regression accelerator based on non-volatile memory according to claim 5, wherein: In step (4), after every two or more iterations, the first control unit determines whether the L2 norm of the residual vector is lower than a threshold.

Images