Deep neural network training method and system for nonvolatile memory chip

Abstract

The invention discloses a deep neural network training method and system for a nonvolatile memory chip, and belongs to the field of deep learning. The method comprises the steps of 1, deploying basic weight data obtained through off-chip training to a cross array of a nonvolatile memory chip; and step 2, carrying out on-chip training on the part of the nonvolatile memory device cross array, which stores the weight data of the last full connection layer in the neural network, by adopting a multi-threshold division optimization weight updating process of the first network training system. According to the method, the weight updating process is optimized by using multi-threshold division, so that the training efficiency is improved, and the reasoning precision of the network-on-chip can be improved to a certain extent by using fewer training rounds.

Description

TECHNICAL FIELD

[0001] The present application relates to a kind of non-volatile storage chip-oriented deep neural network training method and system, belong to deep learning field. BACKGROUND

[0002] With the development of artificial intelligence technology, data grows at an explosive speed, and the demand for computing power continues to rise. The "memory wall" and "power wall" brought by traditional von Neumann architecture have become the key bottleneck for further improvement of chip computing performance. Emerging memory-computing integrated technology solves the overhead problem of data transfer through the integration of storage and computing, and is expected to break through the bottleneck and achieve new breakthroughs in chip computing power and energy efficiency. However, there are still many challenges in performing deep neural network inference and training on memory-computing chips. For example, 1. When deploying a trained neural network on a memory-computing chip, due to the non-ideal nature of the hardware, weight mapping will result in accuracy loss; 2. The nonlinear modulation characteristics of programming operations will cause systematic deviations between actual write values and target conductance values; 3. The coupling effect of array-level parasitic parameters will exacerbate the local distortion of weight distribution, while the time-varying drift characteristics of device conductance further cause dynamic attenuation of stored weights. The above non-ideal hardware constraints accumulate layer by layer when deploying the neural network, eventually causing the statistical distribution of activation values to deviate from the preset range in the software training phase, thereby causing degradation of model inference accuracy.

[0003] To solve the above problems, the current common strategy is to perform on-chip training. One existing on-chip hybrid training mechanism for convolutional neural networks based on memory-computing chips proposes that after deploying the trained model on-chip, the weights of the convolutional layers in the network remain unchanged, and only the weights of the fully connected layers are retrained on-chip to restore the recognition accuracy. However, this method requires the use of iterative write verification technology to repeatedly read the actual conductance of the memory-computing device for verification, resulting in additional hardware circuitry and more time and energy consumption for the implementation of this process, making it difficult to achieve efficient parallelism.

[0004] To solve the above problems, another adaptive non-volatile storage and computing device learning efficient on-chip learning new general algorithm and architecture (see Edge learning using a fully integrated neuro-inspired memristor chip, Science 14 Sep 2023; Vol 381, Issue 6663; pp. 1205-1211; DOI: 10.1126 / science.ade3483) is proposed, which provides an on-chip update scheme that replaces the precise calculation of weight gradient value in the traditional back propagation algorithm with only simple operations involving symbols. Since this scheme is only for the last fully connected layer in the network, the error between the input, output, network prediction result and ideal output of the network is used to calculate the direction of weight update, and a specific voltage pulse programmed in the cross array of the storage and computing device is applied accordingly. Although this scheme does not require write verification and validation, it reduces the energy consumption of on-chip learning, but the single weight update value leads to slow network convergence speed, increases the number of training rounds required, and also causes loss of training accuracy. SUMMARY

[0005] To solve the problem that the single weight update value applied to the cross array of the storage and computing device during on-chip training leads to slow network convergence speed, increases the number of training rounds required, and causes loss of training accuracy, the present application provides a deep neural network training method and system for non-volatile storage and computing chips, which optimizes the weight update process through multi-threshold division, improves the training speed and accuracy, and also improves the network convergence speed. The technical solution is as follows: A deep neural network training method for non-volatile storage and computing chips, comprising the steps of: Step 1, deploying the basic weight data obtained through off-chip training to the cross array of the non-volatile storage and computing chip; Step 2, using the multi-threshold division of the first network training system to optimize the weight update process, and performing on-chip training on the part of the cross array of the non-volatile storage and computing device that stores the weight data of the last fully connected layer in the neural network, to make the on-chip inference more accurate.

[0006] Further, the on-chip training in step 2 includes: Step 21, extracting the input vector of the last fully connected layer in the neural network model deployed on the storage and computing chip for inference, converting the input vector to an output vector through the cross array of the non-volatile storage and computing device, and obtaining a prediction error vector through error calculation from the output vector; Step 22, the input vector, the output vector and the prediction error vector are respectively mapped to the corresponding fixed values set in advance through the corresponding multi-threshold division module; Step 23, the mapped output vector and error vector are operated to obtain an intermediate value; Step 24, different programming pulse groups are applied to the cross array of the non-volatile storage and computing device according to the mapped input vector and the intermediate value.

[0007] Further, the method further comprises step 3, for the new parameters associated with the new extension task of the cross array of the non-volatile storage and computing device, a second network system is used for secondary on-chip training.

[0008] Further, in step 3, the secondary on-chip training comprises: Step 31, the input vector of the last fully connected layer in the network when the neural network model is deployed on the storage and computing chip for inference is extracted, the input vector is converted into a new output vector through the cross array of the non-volatile storage and computing device storing the extended weight data, and the new output vector is converted into a new prediction error vector through error calculation; Step 32, the input vector and the new prediction error vector are respectively mapped to the corresponding fixed values set in advance through the corresponding multi-threshold division module; Step 33, different programming pulse groups are applied to the cross array of the non-volatile storage and computing device according to the two mapped vectors.

[0009] A deep neural network training system for a non-volatile storage and computing chip comprises a first network training system, The first network training system comprises an input vector multi-threshold division module, an error calculation module, an error vector multi-threshold division module, an output vector multi-threshold division module, a weight update value calculation module and a pulse modulation module. The cross array of the non-volatile storage and computing device receives the input vector of the last fully connected layer in the neural network and converts it into an output vector. The error calculation module receives the output vector after error processing and converts it into a prediction error vector. The input vector, the output vector and the prediction error vector are respectively subjected to multi-threshold division through the input vector multi-threshold division module, the output vector multi-threshold division module and the error vector multi-threshold division module to generate mapping fixed values. The weight update calculation module operates the mapping fixed values of the output vector and the mapping fixed values of the prediction error vector to obtain an intermediate value. The pulse adjustment module adjusts the width, direction and amplitude of the output voltage pulse according to the intermediate value and the mapping fixed values of the input vector, so as to apply corresponding voltage pulses to the cross array of the non-volatile storage and computing device.

[0010] Further, it further includes a second network training system, the second network training system includes an input vector multi-threshold division module, an error calculation module, an error vector multi-threshold division module and a pulse modulation module which are the same as the structure of the first network training system, the non-volatile computing device cross array receives the input vector of the last fully connected layer in the neural network, and converts the input vector into a new output vector, at this time, the non-volatile computing device cross array stores the expanded weight data, wherein the array storing the weight directly related to the expanded function part is the training target of the second network training system, the part array storing the basic weight data remains frozen, the error calculation module receives the new output vector after error processing and converts it into a new prediction error vector, the input vector and the new prediction error vector correspond to the new mapping fixed value generated after the input vector multi-threshold division module and the error vector multi-threshold division module are multi-threshold divided, and the pulse modulation module adjusts the width, direction and amplitude of the output voltage pulse according to the new mapping fixed value, so as to apply the corresponding voltage pulse to the device storing the weight related to the expanded function part in the non-volatile computing device cross array.

[0011] Further, the error calculation module performs softmax processing on the output vector, and obtains the prediction error vector by subtracting the target vector.

[0012] Further, two thresholds are respectively arranged in the input vector multi-threshold division module and the output vector multi-threshold division module, three intervals are divided through the setting of the two thresholds, and the input vector falling in the three intervals is mapped to a preset fixed value, and the corresponding vector is recorded.

[0013] Further, two positive thresholds and two negative thresholds are arranged in the error vector multi-threshold division module, five intervals are divided through the setting of the four thresholds, and the prediction error vector falling in the five intervals is mapped to a preset fixed value, and the corresponding vector is recorded.

[0014] Further, the pulse modulation module selects the time of the voltage pulse according to each vector, and selects the direction and amplitude of the voltage pulse according to the positive and negative and absolute value of the value in the vector.

[0015] The present application has the following beneficial effects: In view of the situation that the inference accuracy of the neural network model deployed on the chip is reduced due to the non-ideal factors of the device and circuit, the first network training system is adopted to perform on-chip training on the array of the last fully connected layer in the stored network, and the weight update process is optimized by using multi-threshold division, so that the training efficiency is improved, and the inference accuracy of the on-chip network can be recovered to a certain extent with fewer training rounds; and meanwhile, the conductance modulation characteristics of the non-volatile storage and computing device are combined, and the multi-dimensional design of pulse amplitude and width is used to simplify the calculation process.

[0016] Meanwhile, the second network training system can further expand the function of the current on-chip neural network through secondary on-chip training, so that it can solve new problems of the same type, for example, the network realizing the classification function can be trained to recognize the newly added categories while still being able to recognize the old categories, and the performance balance of new and old tasks is considered, and this method is suitable for on-chip training and dynamic task expansion of the storage and computing integrated chip, and can significantly reduce the hardware complexity and energy consumption. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is a structure diagram of the first network training system of the second embodiment of the present application; Figure 2 is a data stream transmission schematic diagram of the first network training system of the second embodiment of the present application; Figure 3 is a data stream transmission schematic diagram of the second network training system of the second embodiment of the present application; Figure 4 is a part of parameter diagram of the second network training system of the second embodiment of the present application; Figure 5 is a part of parameter diagram of the second network training system of the second embodiment of the present application (explanation principle diagram). DETAILED DESCRIPTION

[0019] In order to make the purpose, technical scheme and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0020] Embodiment one A deep neural network training method for a non-volatile storage and computing chip, comprising the steps of: Step 1, deploying the basic weight data obtained by off-chip training to the cross array of the non-volatile storage and computing chip; Step 2, the first network training system multi-threshold division optimization weight update process is used to perform on-chip training on the part of the non-volatile storage and computing device cross array storing the last fully connected layer weight data in the neural network, so that the accuracy of on-chip inference is recovered.

[0021] Specifically, in step 2, the on-chip training specifically includes the following steps: Step 21, the input vector of the last fully connected layer in the neural network model is extracted when the network is deployed on the storage and computing chip for inference, the input vector is converted into an output vector through the non-volatile storage and computing device cross array, and the output vector is obtained by error calculation to obtain a prediction error vector; Step 22, the input vector, the output vector and the prediction error vector are respectively mapped to the corresponding fixed values set in advance through the corresponding multi-threshold division module; Step 23, the mapped output vector and error vector are operated to obtain an intermediate value; Step 24, different programming pulse groups are applied to the non-volatile storage and computing device cross array according to the mapped input vector and the above intermediate value.

[0022] The network training method of the present application solves the problem that after deploying the ideal weight trained by software outside the chip to the non-volatile storage and computing device cross array, the result obtained by on-chip inference has an unacceptable gap compared with the result of software training outside the chip due to the non-ideal nature of the device and circuit. Therefore, the network training system of the present application needs to be trained on-chip, and the method of the present application not only improves the training efficiency, but also makes the inference accuracy of the on-chip network recover to a certain extent with fewer training rounds; at the same time, combining the conductance modulation characteristics of the non-volatile storage and computing device, the multi-dimensional design of pulse amplitude and width is used to simplify the calculation process.

[0023] Preferably, the above network training method further includes step 3, that is, for the new parameters associated with the expansion task of the non-volatile storage and computing device cross array, a second on-chip training can be selectively performed using the second network system to solve the problem of slow network convergence speed when training for the expansion task.

[0024] Specifically, the second on-chip training includes the following steps: Step 31, the input vector of the last fully connected layer in the neural network model is extracted when the network is deployed on the storage and computing chip for inference, the input vector is converted into a new output vector through the non-volatile storage and computing device cross array storing the expanded weight data, and the new output vector is obtained by error calculation to obtain a new prediction error vector; Step 32, the input vector and the new prediction error vector are respectively mapped to the corresponding fixed values set in advance through the corresponding multi-threshold division module. Step 33, different programming pulse groups are applied to the cross array of the non-volatile computing device according to the two mapped vectors.

[0025] Embodiment two A deep neural network training system for a non-volatile computing chip, as shown in Figure 1 , Figure 2 The system includes a first network training system, which includes an input vector multi-threshold division module, an error calculation module, an error vector multi-threshold division module, an output vector multi-threshold division module, a weight update value calculation module, and a pulse modulation module.

[0026] Specifically, the non-volatile computing device cross array is used to store the weight matrix of the last fully connected layer in the neural network, which can be any network that meets the condition that the last layer is a fully connected layer, such as the classic network model LeNet-5, VGG-11, ResNet-20, etc. The weight matrix is defined as The input vector of the last fully connected layer in the neural network received by the non-volatile computing device cross array is defined as input vector The input vector and the weight matrix are processed by the function to obtain the output vector , and the specific expression is . .

[0027] The error calculation module is connected with the output end of the non-volatile computing device cross array, receives the output vector , and after the output vector is processed by softmax, the difference between the target vector and the prediction error vector is obtained, and the specific expression is .

[0028] The input vector multi-threshold division module simultaneously receives the above input vector . For the vector , the input vector multi-threshold division module is internally provided with two thresholds and , and through the setting of the two thresholds, the values distributed in the three intervals ], ], are respectively mapped to 0, , . For the input vector the input vector after the division and mapping is denoted as vector . .

[0029] The error vector multi-threshold division module is connected with the output end of the error calculation module, and receives the prediction error vector The error vector multi-threshold division module is internally provided with two positive thresholds , and two negative thresholds , The values distributed in the ranges of ], ], ], ], are respectively mapped to , , 0, , For the prediction error vector The input vector after the division and mapping is denoted as vector . .

[0030] The output vector multi-threshold division module is connected with the output end of the cross array of the nonvolatile storage device, and receives the output vector The output vector multi-threshold division module is internally provided with two thresholds and Through the setting of the two thresholds, the values distributed in the ranges of ], ], are respectively mapped to 0, , For the output vector The input vector after the division and mapping is denoted as vector . .

[0031] The weight update value calculation module is arranged between the output vector multi-threshold division module and the error vector multi-threshold division module and the pulse modulation module, and is used to perform operations on the vectors , to obtain an intermediate vector, specifically, the mapped error vector is multiplied by the mapped output vector element by element to obtain the intermediate vector. The expression is: .

[0032] The pulse modulation module receives the intermediate vector from the weight update value calculation module and the mapped input vector from the input vector multi-division module Two items are respectively targeted at the amplitude and width of the voltage pulse used to program the non-volatile computing device, and the pulse modulation module selects the direction and amplitude of the pulse according to the positive and negative and absolute value of the vector, and selects the width of the voltage pulse according to the value of the vector . Since the voltage pulse group used in the conductance modulation of the cross array of the non-volatile computing device needs to be designed from two dimensions of duration and intensity, by targeting the above two items at the two properties of the voltage pulse, the update effect of the weight matrix can be as shown in the following formula: ; both the effect of matrix multiplication of and the effect of avoiding the use of actual hardware to complete the calculation with high overhead are achieved, further reducing the circuit complexity and training power consumption.

[0033] The total controller is connected with the non-volatile computing device cross array, the input vector multi-threshold division module, the error calculation module, the output vector multi-threshold division module, the error vector multi-threshold division module, the weight update value calculation module and the pulse modulation module, and is used for coordinating the work flow of each module.

[0034] The advantage of the first network training system is that by introducing a threshold value to filter out small update values, the oscillation in the network training process is effectively reduced. At the same time, compared with the prior art, which only divides the matrix update value into three values of 0, +1 and -1 according to the sign, the multi-threshold division mapping method used in the system gives the numerical value in the update matrix a distinction in the absolute value size, making the weight update process closer to the ideal training process and reducing the time required for network training.

[0035] Preferably, the above system further comprises a second network training system, as shown in Figure 3 , which includes the same hardware framework as the first network training system, i.e. it includes the same input vector multi-threshold division module, error calculation module, error vector multi-threshold division module and pulse modulation module as the first network training system.

[0036] The non-volatile computing device cross array is used to receive the input vector of the last fully connected layer in the neural network and convert the input vector into a new output vector. At this time, the non-volatile computing device cross array stores the expanded weight data, and the array storing the weight directly related to the expansion function part is the training target of the second network training system, and the part array storing the basic weight data remains frozen. The expanded weight matrix is defined as , and the input vector of the last fully connected layer in the neural network received by the non-volatile computing device cross array is defined as , with the weight matrix is calculated The new output vector is obtained by function processing The specific expression is .

[0037] An error calculation module is connected with the output end of the cross array of the non-volatile computing device, receives the new output vector After the new output vector is processed by softmax , the difference between the target vector and the new output vector is obtained to obtain a new prediction error vector The specific expression is .

[0038] An input vector multi-threshold division module simultaneously receives the input vector For , the input vector multi-threshold division module is internally provided with two thresholds and , and through the setting of the two thresholds, the values distributed in the three intervals ], ] and are respectively mapped to 0, , . For the input vector , the value after the division and mapping is recorded as the vector .

[0039] An error vector multi-threshold division module is connected with the output end of the error calculation module and receives the new prediction error vector The error vector multi-threshold division module is internally provided with two positive thresholds , and two negative thresholds , , and is divided into five intervals, and the values distributed in the intervals ], ], ], ], are respectively mapped to , , 0, , . For the new prediction error vector , the value after the division and mapping is recorded as the vector .

[0040] A pulse modulation module is connected with the output ends of the input vector multi-threshold division module and the error vector multi-threshold division module, and its output end is connected with the cross array of the non-volatile computing device, see Figure 4The output vector is shown The proportion of zero values in the parameters of the relevant items is too large. To prevent the parameters in the calculated weight update matrix from being too sparse and making the network difficult to converge quickly, the output vector The relevant parameters are discarded, and the pulse modulation module only modulates the vector 'and The voltage pulse of the cross array of the non-volatile computing device is designed; that is, the width, i.e., the duration, of the voltage pulse is selected by the vector The width, i.e., the duration, of the voltage pulse is selected by the vector The strength of the output voltage pulse, i.e., the pulse direction and amplitude, is adjusted to apply a corresponding voltage pulse to the cross array of the non-volatile computing device. That is, the update value calculation formula of the weight matrix is: ; The application scenario of the second network training system is the secondary training for new expansion tasks. The specific implementation is to use a new data set to perform on-chip secondary training on the part of the parameters in the model associated with the new expansion task to achieve the expansion of the function. Taking the expansion training task of recognizing new class images as an example, only the part of the parameters in the last fully connected layer associated with the new class to be recognized is trained on-chip. As shown in Figure 4 、 5 The example is that the original eight-class network now wants to implement ten-class classification (two new classes are added, and the original eight classes remain unchanged) on the basis of the original network, that is, the original weight matrix of the last fully connected layer is (1024, 8), and the new weight matrix is (1024, 10). The second network training system only trains the weights of (1024, 2) directly related to the recognition of new classes. This part of the weights has not been trained by software off-chip before and is directly trained on-chip. That is, the devices at the corresponding positions in the array are directly programmed and modulated.

[0041] Since the existing algorithm generally has the problem of slow network convergence speed when training for expansion tasks, the second network training system of embodiment two further optimizes the training process using the multi-threshold non-checking on-chip training system, deletes the sparse parameter items with a too large proportion of zero values in the update matrix calculation formula, so that the network can complete convergence quickly, and the obtained new model can achieve ideal results in new tasks and old tasks. And the deletion of the sparse parameter items further saves the calculation overhead of the element-by-element multiplication operation, improving the inference efficiency.

[0042] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A deep neural network training method for non-volatile memory chips, comprising the following steps: Step 1: Deploy the basic weight data obtained through off-chip training to the cross array of the non-volatile memory chip; Step 2: Using the first network training system, the weight update process is optimized by multi-threshold partitioning. On-chip training is performed on the portion of the non-volatile memory device cross array that stores the weight data of the last fully connected layer in the neural network, so as to improve the accuracy of on-chip inference.

2. The deep neural network training method for non-volatile memory chips according to claim 1, characterized in that, In step 2, the on-chip training includes: Step 21: Extract the input vector of the last fully connected layer in the neural network when the neural network model is deployed on the in-memory computing chip for inference, and convert the input vector into an output vector through the cross array of the non-volatile in-memory computing device. The output vector is used to obtain the prediction error vector through error calculation. Step 22: The input vector, output vector, and prediction error vector are respectively mapped to pre-set fixed values ​​through the corresponding multi-threshold partitioning modules; Step 23: Perform calculations on the mapped output vector and error vector to obtain intermediate values; Step 24: Apply different programming pulse groups to the non-volatile memory device cross array according to the mapped input vector and the intermediate value.

3. The deep neural network training method for non-volatile memory chips according to claim 1, characterized in that, The method further includes step 3, which involves performing secondary on-chip training using a second network system for the new parameters associated with the new extended task of the non-volatile memory cross array.

4. The deep neural network training method for non-volatile memory chips according to claim 3, characterized in that, In step 3, the secondary on-chip training includes: Step 31: Extract the input vector of the last fully connected layer in the neural network when the neural network model is deployed on the in-memory chip for inference. The input vector is converted into a new output vector through the cross array of non-volatile in-memory computing devices that store the expanded weight data. The new output vector is used to calculate the new prediction error vector. Step 32: The input vector and the new prediction error vector are respectively mapped to pre-set fixed values ​​through the corresponding multi-threshold partitioning modules; Step 33: Apply different programming pulse groups to the non-volatile memory device cross array according to the two mapped vectors.

5. A deep neural network training system for non-volatile memory chips, based on the deep neural network training method for non-volatile memory chips according to any one of claims 1 to 4, characterized in that, It includes the first network training system, The first network training system includes: an input vector multi-threshold partitioning module, an error calculation module, an error vector multi-threshold partitioning module, an output vector multi-threshold partitioning module, a weight update value calculation module, and a pulse modulation module. The non-volatile memory device cross array receives the input vector of the last fully connected layer in the neural network and converts it into an output vector. The error calculation module receives the output vector and converts it into a prediction error vector after error processing. The input vector, the output vector, and the prediction error vector are respectively partitioned by the input vector multi-threshold partitioning module, the output vector multi-threshold partitioning module, and the error vector multi-threshold partitioning module to generate a mapping fixed value. The weight update calculation module calculates the mapping fixed value of the output vector and the mapping fixed value of the prediction error vector to obtain an intermediate value. The pulse adjustment module adjusts the width, direction, and amplitude of the output voltage pulse according to the intermediate value and the mapping fixed value of the input vector to apply a corresponding voltage pulse to the non-volatile memory device cross array.

6. A deep neural network training system for non-volatile memory chips according to claim 5, characterized in that, It also includes a second network training system, which includes an input vector multi-threshold partitioning module, an error calculation module, an error vector multi-threshold partitioning module, and a pulse modulation module, all with the same structure as the first network training system. The non-volatile memory device cross array receives the input vector of the last fully connected layer in the neural network and converts the input vector into a new output vector. At this time, the non-volatile memory device cross array stores the expanded weight data. The array storing the weights directly related to the expanded function is the training target of the second network training system, while the array storing the basic weight data remains frozen. The error calculation module receives the new output vector and converts it into a new prediction error vector after error processing. The input vector and the new prediction error vector are respectively processed by the input vector multi-threshold partitioning module and the error vector multi-threshold partitioning module to generate new mapping fixed values. The pulse modulation module adjusts the width, direction, and amplitude of the output voltage pulse according to the new mapping fixed value to apply the corresponding voltage pulse to the device storing the weights related to the expanded function in the non-volatile memory device cross array.

7. A deep neural network training system for non-volatile memory chips according to claim 5, characterized in that, The error calculation module performs softmax processing on the output vector and subtracts it from the target vector to obtain the prediction error vector.

8. A deep neural network training system for non-volatile memory chips according to claim 5, characterized in that, The input vector multi-threshold partitioning module and the output vector multi-threshold partitioning module each set two thresholds. By setting the two thresholds, the input vector is divided into three intervals, and the input vector falling into these three intervals is mapped to a preset fixed value, which is denoted as the corresponding vector.

9. A deep neural network training system for non-volatile memory chips according to claim 5, characterized in that, The error vector multi-threshold division module sets two positive thresholds and two negative thresholds. By setting the four thresholds, the system divides the system into five intervals and maps the prediction error vectors falling within these five intervals to preset fixed values, which are denoted as the corresponding vectors.

10. A deep neural network training system for non-volatile memory chips according to claim 5, characterized in that, The pulse modulation module selects the duration of the voltage pulse according to each of the vectors, and selects the direction and amplitude of the voltage pulse according to the sign and absolute value of the values ​​in the vectors, respectively.

Images