Precision compensation device and method for temperature drift characteristics of in-memory computing device

Abstract

The application provides a precision compensation device and method for temperature drift characteristics of a memory-computing integrated device. The precision compensation device comprises an upper computer, a compensation array module, a partial sum compensation logic module, a temperature sensing module, a compensation array control module, an ADC array and an addition tree. The compensation array module is used for storing weight values trained for different temperature ranges and performing first compensation on an inference result. The partial sum compensation logic module is used for sampling an output result of the addition tree and performing second compensation on the inference result. The application can solve problems such as deviation of vector matrix multiplication calculation results and degradation of neural network inference precision caused by weight drift of the memory-computing integrated device with temperature, and can be applied to different types of memory-computing integrated devices and has certain universality.

Description

Technical Field

[0001] This invention relates to a calculation accuracy compensation device and method for in-memory computing devices with temperature drift characteristics, belonging to the field of artificial intelligence technology. Background Technology

[0002] In recent years, deep neural networks have achieved tremendous success in various machine learning applications, such as image processing, language translation, and speech recognition. However, the large amount of data movement between computing units and off-chip memory poses a significant challenge to traditional AI accelerators based on the von Neumann architecture. In-memory AI accelerators, by fusing computation and storage, break away from the traditional von Neumann architecture and can effectively alleviate the memory wall problem.

[0003] AI accelerators based on in-memory computing technology can store neural network weights within an array of in-memory computing devices and perform large-scale multiply-accumulate operations in parallel during neural network inference. This significantly improves the computational efficiency of neural networks on hardware, minimizes the cost of data movement, and substantially improves the energy efficiency of in-memory computing chips.

[0004] However, AI accelerators based on in-memory computing technology suffer from device temperature drift. The programmed values ​​of in-memory computing devices shift with temperature changes, meaning that the neural network weights mapped in the in-memory array are affected by temperature. This shift in neural network weights leads to a significant decrease in inference accuracy during neural network operation, even degrading to extremely low random values. Therefore, it is necessary to address the degradation of neural network inference accuracy caused by temperature drift in in-memory computing devices to achieve the expected AI performance of in-memory computing AI accelerators. Summary of the Invention

[0005] To address the issues of deviation in vector matrix multiplication calculation results and degradation of neural network inference accuracy caused by temperature drift in in-memory computing devices, and to further improve the performance and engineering practicality of in-memory computing chips, this invention proposes a general-purpose accuracy compensation device and method.

[0006] The technical solution adopted by the device of the present invention is as follows:

[0007] A precision compensation device for the temperature drift characteristics of in-memory computing devices includes an in-memory computing array and: a compensation array module for storing weight values ​​trained for different temperature ranges and performing the first compensation on the inference results; a temperature sensing module for acquiring the operating temperature of the compensation array module and converting the analog temperature into a digital control quantity, which is then sent to the compensation array control module; a compensation array control module for activating the corresponding part of the compensation array module at different operating temperatures based on the output of the temperature sensing module; an ADC array for converting the analog current outputs of the in-memory computing array and the compensation array module into digital quantities; an adder tree for accumulating the digital quantities output by the ADC array and outputting the result to a partial sum compensation logic module; a partial sum compensation logic module for sampling the output of the adder tree and performing a second compensation on the inference results; and a host computer for updating the weights of the in-memory computing array and the compensation array module, and interacting with the partial sum compensation logic module and the temperature sensing module.

[0008] Furthermore, the compensation array module includes four memory computing device arrays for storing weight values ​​trained for four different temperature ranges. The memory computing device arrays are composed of any type of memory computing device affected by temperature drift in the form of a crossbar array. The input of the compensation array module is an externally input voltage excitation, and its output is the multiplication and accumulation result of the four memory computing device arrays.

[0009] Furthermore, the four memory device arrays are specifically as follows: memory device array T1 stores the weights for compensation accuracy in the range of -40 to 0℃, memory device array T2 stores the weights for compensation accuracy in the range of 25 to 50℃, memory device array T3 stores the weights for compensation accuracy in the range of 50 to 70℃, and memory device array T4 stores the weights for compensation accuracy in the range of 70 to 90℃.

[0010] Furthermore, the compensation array control module includes: a decoder for decoding the digital control quantity output by the temperature sensing module, thereby controlling the column selector; and a column selector for selecting the compensation column output corresponding to the temperature range according to the output signal of the decoder. The input of the compensation array control module is the output of the compensation array module and the output of the temperature sensing module, and the output of the compensation array control module is the portion of the compensation array selected according to the digital control quantity.

[0011] Furthermore, the partial sum compensation logic module includes: an input sampler for sampling the partial sum output of the addition tree and then sending it to the host computer; a compensation factor storage unit for storing the compensation factor sent by the host computer; and a multiplier for multiplying the compensation factor with the partial sum to complete the compensation and outputting it to the next layer. The input of the partial sum compensation logic module is the multiplied-accumulated partial sum after digital accumulation through the addition tree, as well as the compensation factor and the sampling signal sent by the host computer. Its output is the multiplied-accumulated partial sum after factor compensation and the multiplied-accumulated partial sum sent to the host computer after sampling without factor compensation.

[0012] The method of the present invention utilizing the above-mentioned accuracy compensation device for temperature drift characteristics of in-memory computing devices includes the following steps:

[0013] (1) The host computer first sends a set of input excitation weights to the in-memory computing array and the compensation array module. At this time, the in-memory computing array completes the vector matrix multiplication and accumulation calculation of the neural network. At the same time, the temperature sensing module collects the current working temperature of the array and outputs the temperature digital control quantity to the compensation array control module. The compensation array control module selects a set of compensation array outputs representing the corresponding temperature range according to the digital control quantity, and reads them together with the output of the in-memory computing array by the ADC array and converts them into digital quantities.

[0014] (2) The addition tree accumulates the output of the ADC array and outputs a partial sum to the partial sum compensation logic module. At this time, if the partial sum compensation logic module has not stored a set of compensation factors, the host computer sends a sampling signal to the partial sum compensation logic module. The partial sum compensation logic module samples the input partial sum and sends it to the host computer. Then the host computer calculates the compensation factor by combining the ideal output and the actual output, and sends the compensation factor to the partial sum compensation logic module until the compensation factor corresponding to each layer of the neural network is stored in the register of the partial sum compensation logic module.

[0015] (3) After storing a set of compensation factors for the corresponding neural network, the inference of the neural network is formally started and the inference results are compensated; the compensation array module performs the first compensation on the output of the in-memory computing array by deploying neural network weights corresponding to the working temperature range, while the partial compensation logic module performs the second compensation by multiplying the inference results of each layer of the neural network by the compensation factor.

[0016] This invention utilizes another method of the aforementioned precision compensation device for temperature drift characteristics of in-memory computing devices, comprising the following steps: The host computer receives the temperature digital control signal from the temperature sensing module. If it detects that the operating temperature exceeds the compensation temperature range, and the difference between the ideal output and the actual output increases to exceed the set threshold, a software precision recovery process is triggered. That is, the host computer refreshes the weights of the in-memory computing array, updating the drifted weights to a set of non-drifted weights. At this time, the arrays in the compensation array module are not selected, and all compensation factors are set to 1. After each calculation is completed, the calculation result is sent to the host computer for judgment. If the error exceeds the threshold, the weights are updated again.

[0017] This invention provides a precision compensation device for the temperature drift characteristics of in-memory computing devices. It can compensate for the precision of in-memory computing arrays composed of different types of devices (e.g., opto-in-memory units, RRAM arrays, PCM arrays, FLASH arrays, etc.), demonstrating strong versatility. The invention first uses additional weights to perform initial compensation on the neural network's multiplication and accumulation results. Then, it obtains a compensation factor through calibration with a host computer and multiplies the calculation result by this factor for further compensation, thus addressing most cases where precision loss is controllable. Simultaneously, a software recovery method is used to refresh the weights to address cases where precision loss is uncontrollable. This compensation device structure can significantly mitigate the temperature-dependent degradation of neural network inference precision, effectively improving the performance of neural networks on in-memory computing chips. Attached Figure Description

[0018] Figure 1 This is the overall architecture diagram of a precision compensation device for the temperature drift characteristics of in-memory computing devices. Input_0~i represents the input stimulus, MAC represents the calculation result of the in-memory computing device array, S1~S4 represents the calculation result of the compensation array, Temp_Code is the temperature control code, Partial Sum is the output result of the addition tree, ACT_OUT is the sampling of Partial Sum by the partial sum compensation logic module, CFs is the compensation factor sent from the host computer to the partial sum compensation logic module, and SAMPLE is the sampling signal sent from the host computer to the partial sum compensation logic module.

[0019] Figure 2 This is a flowchart illustrating the operation of the device of the present invention.

[0020] Figure 3 This is a schematic diagram of the internal structure of the temperature sensing module. Here, T represents the array's operating temperature, and Temp_Code is the temperature control code output by this module.

[0021] Figure 4This is a schematic diagram of the internal structure of the compensation array control module. Temp_Code is the temperature control code input to the temperature sensing module, SEL_Code is the decoding result of the temperature control code by the decoder, and S1 to S4 represent the calculation results input to the compensation array module.

[0022] Figure 5 This is a schematic diagram of the internal structure of the partial sum compensation logic module. Partial Sum is the output of the addition tree, ACT_OUT is the sampling of Partial Sum by the partial sum compensation logic module, CFs are the compensation factors sent from the host computer to the partial sum compensation logic module, and CF1 to n are the compensation factors already stored in the compensation factor storage unit. Detailed Implementation

[0023] The in-memory computing device in this embodiment uses an opto-in-memory computing unit. The specific structure of the computing unit can use the structure of Embodiments 1, 2, or 3 in patent application CN201910415827.1. The specific structure of the in-memory computing neural network array composed of opto-in-memory computing units can use the structure of Embodiment 4 in patent application CN201910416021.4 or Embodiment 1 in patent application CN202010322172.6, etc.

[0024] The accuracy compensation device of the present invention is as follows: Figure 1 As shown. The compensation array module stores weight values ​​trained for different temperature ranges and performs the first compensation on the inference results. The compensation array control module activates the corresponding parts of the compensation array module for different temperature ranges based on the output of the temperature sensing module. The temperature sensing module collects the operating temperature of the compensation array module and converts the analog temperature into a digital control quantity, sending it to the compensation array control module and the host computer. The partial sum compensation logic module is used to sample the partial sum during software recovery and to perform the second compensation during hardware compensation. The ADC array converts the analog current outputs of the in-memory computing array and the compensation array into digital quantities. The adder tree accumulates the digital quantities output by the ADC array to form the partial sum, which is then output to the partial sum compensation logic module.

[0025] The compensation array module includes T1-T4 memory-in-memory (MIM) arrays, used to store weight values ​​trained for different temperature ranges. For example, array T1 stores weights for compensation accuracy within the -40 to 0°C range, array T2 stores weights for compensation accuracy within the 25 to 50°C range, array T3 stores weights for compensation accuracy within the 50 to 70°C range, and array T4 stores weights for compensation accuracy within the 70 to 90°C range. These arrays can be constructed from any type of MIM device affected by temperature drift in the form of a crossbar array. Besides the opto-MIM array in this embodiment, RRAM, Flash, etc., can also be used. The temperature range corresponding to the weights of the T1-T4 compensation arrays is set and adjusted by the host computer and trained within the corresponding temperature range. The input to the compensation array module is the external voltage excitation; the module output is the multiplicative and accumulator result of the corresponding T1-T4 compensation arrays.

[0026] The compensation array control module includes: a decoder, used to decode the output control code of the temperature sensing module, thereby controlling the column selector; and a column selector, which selects the compensation column output corresponding to the temperature range based on the output signal of the decoder. The inputs to the compensation array control module are the output of the compensation array and the output of the temperature sensing module, and the output is the portion of the compensation array selected according to the digital control quantity.

[0027] The partial sum compensation logic module includes: an input sampler for sampling the partial sum output of the addition tree and then sending it to the host computer; a compensation factor storage unit for storing the compensation factors sent by the host computer; and a multiplier for multiplying the compensation factors by the partial sum to complete the compensation output to the next layer. The inputs to the partial sum compensation logic module are the multiplied-accumulated partial sum after digital accumulation by the addition tree, the compensation factors sent by the host computer, and the sampled signal. Its outputs are the multiplied-accumulated partial sum after factor compensation and the sampled multiplied-accumulated partial sum sent to the host computer without factor compensation.

[0028] The working process of this invention's device is divided into two parts: software recovery and hardware compensation. Software recovery is used when the error caused by temperature drift is too large and the accuracy loss is uncontrollable. Accuracy is restored by reprogramming the array weights through a host computer. Hardware compensation is used when the accuracy loss is controllable, and accuracy is compensated through specific hardware units. Repeated programming of the device array can shorten the device's lifespan; therefore, hardware compensation is used in most cases to extend the chip's service life. These two compensation strategies are adaptively and flexibly adjusted by the host computer software in conjunction with the hardware compensation logic, adapting to different degrees of accuracy loss. The specific workflow is as follows: First, the compensation temperature range is determined. If the array operating temperature is within the compensation temperature range (i.e., the temperature range covered by compensation array modules T1 to T4), the hardware compensation strategy is directly adopted. If the array operating temperature is not within the compensation temperature range, the error of the output result needs further judgment. If the error is greater than the threshold set by the host computer, the software recovery process is initiated. That is, the host computer refreshes the array weights, updating the drifted weights to a set of undrifted weights to restore accuracy.

[0029] Example 1

[0030] This embodiment describes the hardware compensation process. The partial sum compensation logic module interacts with the host computer and performs a second compensation. The host computer stores the compensation factors for the partial sums of each layer of the neural network in this module, and performs a second compensation on the partial sums when the neural network completes the partial sum calculation.

[0031] The compensation array module takes the external voltage excitation Input_i as input and outputs the neural network calculation results S1-S4 after the first compensation is completed through an additional weight array. Arrays T1-T4 correspond to different temperature ranges. The temperature sensing module is shown in the attached diagram. Figure 2 As shown, the input is the array operating temperature T, and the output is the digital control quantity Temp_code. The compensation array control module is shown in the attached diagram. Figure 3 As shown, the inputs are the digital control quantity Temp_code output from the temperature sensing module and the multiplication and accumulation results S1-S4 from the compensation array module. The output is a multiplication and accumulation result S1 / S2 / S3 / S4 selected based on the digital control quantity Temp_code. Partial compensation logic modules are shown in the attached diagram. Figure 4 As shown, the inputs include the output of the addition tree (Partial Sum), the sampling signal SAMPLE sent to the module by the host computer, and the compensation factor CFs. The outputs are the actual output ACT_OUT sent by the input sampler to the host computer, and the neural network calculation results after completing the second compensation.

[0032] Applying the aforementioned precision compensation device to the in-memory computing chip, the first compensation is performed through the weights configured in the compensation array module. The host computer first sends a set of calibration inputs, Input_i, to the in-memory computing array. Then, a heating device heats the array, ensuring it operates within a defined temperature range (in this embodiment, the array operating temperature is assumed to be 50–70°C, corresponding to array T2). The in-memory computing array performs vector matrix multiplication and accumulation calculations for the neural network, obtaining the output MAC, which is then input to the ADC array. Simultaneously, the temperature sensing module detects the array's operating temperature and outputs a digital control quantity, Temp_Code, to the compensation array control module. The compensation array control module selects the output S2 of array T2 and sends it to the ADC array based on the control quantity. After analog-to-digital conversion by the ADC, MAC and S2 are input to the addition tree, which sums the calculation results of each column. This completes the first compensation of the calculation results.

[0033] Before performing the second compensation, the device needs to calculate the compensation factor. After the neural network weights are deployed on the in-memory computing chip, the compensation factor only needs to be calculated and deployed once before being stored in the chip. The output of the addition tree, Partial Sum, is input to the partial sum compensation logic module. Before obtaining the compensation factor, the host computer sends a SAMPLE signal to this module, causing the input sampler to sample the Partial Sum and send the sampling result ACT_OUT to the host computer. ACT_OUT is the actual Partial Sum before the second compensation, while the host computer stores the ideal output result, from which the compensation factor CFs can be calculated.

[0034] After the host computer calculates the compensation factor, it sends it to the compensation factor storage unit for storage. After storing a set of compensation factors for the corresponding neural network model, the entire device can officially start the neural network inference and compensate for the inference results.

[0035] The compensation array performs the first compensation on the output of the in-memory computing device array by deploying additional neural network weights corresponding to the operating temperature range, while the partial sum compensation logic module performs the second compensation on the partial sum by multiplying the inference result of each layer of the neural network by the compensation factor, and then sends the compensation result to the next layer.

[0036] Example 2

[0037] When the operating temperature exceeds the compensation temperature range of the compensation array in Example 1 and the accuracy loss becomes uncontrollable, the host computer will invoke the software recovery process of this example to restore the accuracy. The partial sum compensation logic module is used during the software recovery process to sample the partial sum, interact with the host computer, and output the recovery result.

[0038] The in-memory computing device array takes an external voltage excitation Input_i as input and outputs the calculation result MAC. The temperature sensing module is shown in the attached diagram. Figure 2 As shown, the input is the array operating temperature T, and the output is the digital control quantity Temp_code. Partial compensation logic modules are shown in the attached diagram. Figure 4 As shown, the inputs include the output of the addition tree (Partial Sum), the sampling signal SAMPLE sent to the module by the host computer, and the compensation factor CFs. The outputs are the actual output ACT_OUT sent by the input sampler to the host computer, and the neural network calculation results after completing the second compensation.

[0039] The host computer first sends a set of inputs, Input_i, to the in-memory computing array to obtain the calculation result, MAC. The temperature sensing module detects the array's operating temperature and outputs a digital temperature control value, Temp_Code. If the temperature exceeds the compensation range (e.g., 130℃), the compensation array modules T1 through T4 are not selected. Simultaneously, the host computer receives the digital temperature control value from the temperature sensing module, decodes it, and finds that the temperature exceeds the compensation range. It then sets all compensation factors, CFs, to 1 and sends them to the partial sum compensation logic module. At this time, the partial sum logic module samples the Partial Sum output from the adder tree to form ACT_OUT and sends it to the host computer.

[0040] The host computer calculates the error between the ideal output and the actual output, and determines whether the error exceeds a user-set threshold. If the error exceeds the threshold, the weights of the in-memory computing device array are updated. Subsequently, the host computer continues to calculate the error between the ideal output and the actual output until the error exceeds the threshold again, at which point the weights are updated.

Claims

1. A precision compensation device for temperature drift characteristics of an in-memory computing device, comprising an in-memory computing array, characterized in that, The precision compensation device further comprises: a compensation array module for storing weight values trained for different temperature ranges and performing first compensation on the inference result; a temperature sensing module for collecting the working temperature of the compensation array module and converting the analog temperature into a digital control quantity, and sending the digital control quantity to the compensation array control module; a compensation array control module for activating the corresponding part of the compensation array module at different working temperatures according to the output of the temperature sensing module; an ADC array for converting the analog current output of the compute-in-memory array and the compensation array module into a digital quantity; an addition tree for accumulating the digital quantity output by the ADC array and outputting the result to the partial sum compensation logic module; a partial sum compensation logic module for sampling the output result of the addition tree and performing second compensation on the inference result; a host computer for updating the weight of the compute-in-memory array and the compensation array module, interacting with the partial sum compensation logic module, and interacting with the temperature sensing module; the compensation array module comprises four compute-in-memory device arrays for storing weight values trained for four different temperature ranges, and the compute-in-memory device array array is formed in the form of a crossbar array by any compute-in-memory device affected by temperature drift; the input of the compensation array module is the voltage excitation input from outside, and the output is the multiply-accumulate result of the four compute-in-memory device arrays; the four compute-in-memory device arrays are specifically: compute-in-memory device array T1 stores weights for compensating precision in the range of -40-0℃, compute-in-memory device array T2 stores weights for compensating precision in the range of 25-50℃, compute-in-memory device array T3 stores weights for compensating precision in the range of 50-70℃, and compute-in-memory device array T4 stores weights for compensating precision in the range of 70-90℃.

2. The precision compensation device for temperature drift characteristics of the in-memory computing integrated device according to claim 1, wherein The compensation array control module comprises: a decoder for decoding the digital control quantity output by the temperature sensing module to control the column selector; a column selector for selecting the compensation column output of the corresponding temperature range according to the output signal of the decoder; The input of the compensation array control module is the output of the compensation array module and the output of the temperature sensing module, and the output of the compensation array control module is the part of the compensation array selected according to the digital control quantity.

3. The precision compensation device for temperature drift characteristics of the in-memory computing integrated device according to claim 1, wherein The partial sum compensation logic module comprises: an input sampler for sampling the partial sum output of the addition tree and then sending it to the host computer; a compensation factor storage unit for storing the compensation factor sent by the host computer; a multiplier for multiplying the compensation factor and the partial sum to complete the compensation and output to the next layer; The input of the partial sum compensation logic module is the multiply-accumulate partial sum after digital accumulation by the addition tree and the compensation factor and the sampling signal sent by the host computer, and the output is the multiply-accumulate partial sum after completion of the factor compensation and the multiply-accumulate partial sum after sampling sent to the host computer without factor compensation.

4. The method of claim 1, wherein the precision compensation device is used for the temperature drift characteristics of the in-memory computing device. The method comprises the following steps: (1) The host computer first sends a set of input excitation weights to the memory-computing integrated array and the compensation array module. At this time, the memory-computing integrated array completes the vector matrix multiplication and accumulation calculation of the neural network. At the same time, the temperature sensing module collects the working temperature of the current array and outputs the temperature digital control quantity to the compensation array control module. The compensation array control module selects a set of compensation array outputs representing the corresponding temperature range according to the digital control quantity, and the outputs of the memory-computing integrated array are read out by the ADC array and converted into digital quantities; (2) The addition tree accumulates the outputs of the ADC array and outputs partial sums to the partial sum compensation logic module. At this time, if a set of compensation factors is not stored in the partial sum compensation logic module, the host computer sends a sampling signal to the partial sum compensation logic module. The partial sum compensation logic module samples the input partial sum and sends it to the host computer. Then the host computer calculates the compensation factor combining the ideal output and the actual output, and sends the compensation factor to the partial sum compensation logic module until the corresponding compensation factor of each layer of the neural network is stored in the register of the partial sum compensation logic module; (3) After storing a set of compensation factors corresponding to the neural network, the inference of the neural network is formally started and the inference result is compensated. The compensation array module compensates the output of the memory-computing integrated array by deploying the neural network weight corresponding to the working temperature range for the first time, and the partial sum compensation logic module compensates the inference result of each layer of the neural network by multiplying the compensation factor for the second time.

5. The method of claim 1, wherein the precision compensation device is used for the temperature drift characteristics of the in-memory computing device. The method comprises the following steps: the host computer receives the temperature digital control quantity of the temperature sensing module. If it is found that the working temperature exceeds the compensation temperature range, and the difference between the ideal output and the actual output increases to exceed the set threshold, the software precision recovery process is triggered, that is, the weights of the memory-computing integrated array are refreshed by the host computer to update the drifted weights to a set of non-drifted weights. After each calculation, the calculation result is sent to the host computer for judgment. If the error exceeds the threshold, the weights are updated again.

6. The method of claim 5, wherein the precision compensation device is used for the temperature drift characteristics of the in-memory computing device. When the software precision recovery process is triggered, none of the arrays in the compensation array module is selected, and the compensation factor is set to 1.

Images