Multi-layer convolutional pooling sensory computing integrated system based on hall strip array

Abstract

The application discloses a multi-layer convolutional pooling sensing and computing integrated system based on a Hall strip array, which uses a sensing array as a data acquisition end, uses a controller to control an array gating storage module to collect sensing voltage, outputs a current signal through a signal conversion module, and realizes hardware convolutional activation pooling operation under the coordination of the controller by utilizing the sensing and computing integrated property of the Hall strip device, so that analog output is obtained through multi-layer hardware convolutional activation pooling operation. The application directly outputs an operation result from an analog signal from the sensing array, has low power consumption, high speed and ADC sampling free, can reduce the data carrying and conversion process, and can provide efficient edge end calculation for devices in the Internet of Things field.

Description

Technical Field

[0001] This invention belongs to the field of sensor-memory-computing integration, specifically based on a multi-layer convolutional pooling sensor-memory-computing integration system using Hall strip arrays. Background Technology

[0002] In traditional computer design, storage modules primarily serve computing, thus the design considers the separation and prioritization of storage and computing. Today, storage and computing must be considered holistically to optimally coordinate data acquisition, transmission, and processing. This redistribution process between storage and computing faces various challenges, mainly manifested as storage walls, bandwidth walls, and power consumption walls. With the development of cloud computing and artificial intelligence (AI) applications in recent years, slow data movement and high energy consumption during data transfer have become key bottlenecks in computing, especially given the massive data deluge in data centers.

[0003] If the system can directly process information collected by edge sensors, it can significantly improve the speed of convolution operations, reduce data movement during computation, and retain the necessary feature data. Such a system can solve the data movement and power consumption problems of traditional von Neumann architectures. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a multi-layer convolutional pooling sensing-memory-computing integrated system based on Hall bar arrays. This system architecture uses a sensing array as the data acquisition end and a controller to perform gating control on the array and control of each link. Based on Hall bar devices that integrate storage and computation, convolutional activation pooling of data is implemented using analog circuits, thus designing an integrated sensing-memory-computing system.

[0005] The multi-layer convolutional pooling sensing-in-memory computing system based on Hall bar arrays includes a sensing array, an array gating and storage module, a signal conversion module, a Hall bar array convolution module, an activation pooling storage array module, and a controller.

[0006] The sensor array serves as a data acquisition module for analog signals, receiving different sensor signals and outputting voltage signals.

[0007] The array gating and storage module stores the voltage signal output by the sensor array through a voltage storage hardware circuit with a controllable gating path, and outputs the voltage value corresponding to the convolution kernel in the convolution window.

[0008] Preferably, the array gating storage module further includes an operational amplifier module, which is used to amplify the voltage output by the sensor array before inputting it into the storage voltage circuit.

[0009] The voltage storage hardware circuit with a controllable selection path is used to realize the data storage of selected rows and columns. It includes a first amplifier OA1, diodes D1 and D2, a second amplifier OA2, resistors R1 and R2, a capacitor C, and a field-effect transistor M1. The output terminal of the first amplifier OA1 is connected to the anode of diode D1. The cathode of diode D1 is connected to the non-inverting input terminal of the second amplifier OA2. The anode of diode D2 is connected to the inverting input terminal of the first amplifier OA1. The cathode of diode D2 is connected to the anode of diode D1. One end of resistor R2 is connected to the inverting input terminal of the first amplifier OA1, and the other end is connected to both the output and inverting input terminals of the second amplifier OA2. One end of resistor R1 is connected to the non-inverting input terminal of the first amplifier OA1, and the other end is grounded. The two ends of capacitor C are connected to the source of field-effect transistor M1 and the non-inverting input terminal of the second amplifier OA2, respectively. The drain of the field-effect transistor M1 is connected to the non-inverting input of the second amplifier OA2, the gate is connected to the storage discharge control signal Vs output by the FPGA, and the source is grounded.

[0010] The signal conversion module receives the voltage value output by the array gating and storage module, and converts the voltage value into a current signal for the Hall bar corresponding to the convolution kernel.

[0011] Preferably, the amplitude of the current signal output by the signal conversion module does not exceed the critical current value that changes the Hall resistance value.

[0012] Preferably, the signal conversion module is used to convert a voltage signal into a corresponding current signal, and includes a third amplifier OA3, resistors R3, R4, R5, R6, and R7, a Schottky diode D3, and a fourth amplifier OA4. The output terminal of the third amplifier OA3 is connected to one end of resistor R5. The other end of resistor R5 is connected to the non-inverting input terminal of the fourth amplifier OA4. One end of resistor R4 is connected to the inverting input terminal of the third amplifier OA3, and the other end is connected to the output terminal of the third amplifier OA3. The two ends of resistor R6 are connected to the inverting input terminal of the third amplifier OA3 and ground, respectively. The output terminal of the fourth amplifier OA4 is connected to one end of resistor R7. The other end of resistor R7 is connected to the non-inverting input terminal of the third amplifier OA3. The output terminal of the fourth amplifier OA4 is connected to the inverting input terminal of the fourth amplifier OA4. The non-inverting input terminal of the fourth amplifier OA4 is connected to the positive terminal of the Schottky diode D3. The voltage input Vin is connected to resistor R3. The current output Iout is output from the negative terminal of the Schottky diode D3.

[0013] The Hall bar array convolution module includes a Hall bar array composed of multiple Hall bars, an operational amplifier, and an analog multiply-accumulate circuit; the Hall bar array receives the current signal I output from the signal conversion module. N The analog multiply-accumulate circuit converts the anomalous Hall resistance R of the Hall bar.hallbarN The operational amplifier gain A is used as the multiplicand, and the current signal is used as the multiplier. All anomalous Hall voltages after dot product are added together, and the result of the convolution operation is output.

[0014] The activation pooling storage array is used to implement ReLU activation and max pooling storage. The result of the convolution operation is used as input, and the pooling voltage is stored in the storage array module. The storage array module is the storage circuit of the array gating storage module. The capacitor C is used to store the voltage. The storage voltage in the storage array is input to the next layer Hall bar array convolution activation pooling storage module through the action of the controller. After passing through multiple series-connected signal conversion modules, Hall bar array convolution modules, and activation pooling storage array modules, the result is directly output in the form of analog quantity.

[0015] The controller acts as a scheduling module for each stage. Convolutional kernels are obtained through training, and the controller writes them into the Hall bar array, resulting in the anomalous Hall resistance values ​​of the Hall bars.

[0016] The present invention has the following beneficial effects:

[0017] 1. The array gating and storage module stores the voltage values ​​collected by the sensor array through a voltage storage hardware circuit with a controllable gating path, outputs the voltage value corresponding to the convolution kernel in the convolution window, and directly collects data from the sensor and inputs it into the Hall bar array according to the corresponding convolution kernel, thereby reducing the data handling process and high power consumption.

[0018] 2. The signal conversion module receives the voltage signal output by the array gating module and converts the voltage signal into a current signal for the Hall bar corresponding to the convolution kernel. Through the signal conversion module, in-memory computation can be directly performed using the Hall bar array, and the data source is the sensor array, realizing a multi-layer convolutional pooling in-memory computing system integrating sensors and Hall bar arrays.

[0019] 3. The aforementioned multi-layer convolutional pooling sensing-in-memory computing system based on Hall bar array uses the sensor as the data input source. After passing through the signal conversion module and the multi-layer Hall bar array convolutional activation pooling module, the analog signal is directly output, eliminating the need for traditional DAC and ADC steps. Attached Figure Description

[0020] Figure 1 This is a system block diagram of the multi-layer convolutional pooling sensing-in-memory computing system architecture based on Hall bar array in the embodiment;

[0021] Figure 2 A circuit diagram of the array gating memory module in the embodiment;

[0022] Figure 3This is a circuit diagram of the signal conversion module;

[0023] Figure 4 The variation of the anomalous Hall resistance of the Hall bar under pulsed current driving is shown.

[0024] Figure 5 This is a schematic diagram of the convolution operation module in the embodiment;

[0025] Figure 6 This is a flowchart of a multi-layer convolutional pooling sensing-in-memory computing system based on Hall bar arrays. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further explained and described below in conjunction with the accompanying drawings. It must be noted that the embodiments described below are only some embodiments of this invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the protection scope of this invention.

[0027] like Figure 1 As shown, a multi-layer convolutional pooling sensing-in-memory computing system based on Hall bar arrays is presented.

[0028] The multi-layer convolutional pooling sensing-in-memory computing system based on Hall bar arrays includes a sensing array, an array gating and storage module, a signal conversion module, a Hall bar array convolution module, an activation pooling storage array module, and a controller.

[0029] The sensor array serves as a data acquisition module for analog signals. Different sensors can receive different sensing signals, such as pressure, light intensity, sound, etc., and output voltage signals.

[0030] The array gating and storage module stores the voltage values ​​acquired by the sensor array through a voltage storage hardware circuit with a controllable gating path, and outputs the voltage value corresponding to the convolution kernel in the convolution window. The array gating and storage module also includes an operational amplifier module, used to amplify the voltage acquired by the sensor array by a certain factor before inputting it into the storage voltage circuit.

[0031] like Figure 2As shown, the voltage storage hardware circuit with a controllable selection path is used to realize the data storage of selected rows and columns. It includes a first amplifier OA1, diodes D1 and D2, a second amplifier OA2, resistors R1 and R2, a capacitor C, and a field-effect transistor M1. The output terminal of the first amplifier OA1 is connected to the anode of diode D1. The cathode of diode D1 is connected to the non-inverting input terminal of the second amplifier OA2. The anode of diode D2 is connected to the inverting input terminal of the first amplifier OA1. The cathode of diode D2 is connected to the anode of diode D1. One end of resistor R2 is connected to the inverting input terminal of the first amplifier OA1, and the other end is connected to both the output and inverting input terminals of the second amplifier OA2. One end of resistor R1 is connected to the non-inverting input terminal of the first amplifier OA1, and the other end is grounded. The two ends of capacitor C are connected to the source of field-effect transistor M1 and the non-inverting input terminal of the second amplifier OA2, respectively. The drain of the field-effect transistor M1 is connected to the non-inverting input of the second amplifier OA2, the gate is connected to the storage discharge control signal Vs output by the FPGA, and the source is grounded.

[0032] The signal conversion module receives the voltage signal output by the array gating and storage module, and converts the voltage signal into a current signal for the Hall bar corresponding to the convolution kernel. The amplitude of the current signal output by the signal conversion module cannot exceed the critical current value that changes the Hall resistance.

[0033] like Figure 3 As shown, the signal conversion module is used to convert a voltage signal into a corresponding current signal. It includes a third amplifier OA3, resistors R3, R4, R5, R6, and R7, a Schottky diode D3, and a fourth amplifier OA4. The output of the third amplifier OA3 is connected to one end of resistor R5. The other end of resistor R5 is connected to the non-inverting input of the fourth amplifier OA4. One end of resistor R4 is connected to the inverting input of the third amplifier OA3, and the other end is connected to the output of the third amplifier OA3. The two ends of resistor R6 are connected to the inverting input of the third amplifier OA3 and ground, respectively. The output of the fourth amplifier OA4 is connected to one end of resistor R7. The other end of resistor R7 is connected to the non-inverting input of the third amplifier OA3. The output of the fourth amplifier OA4 is connected to its inverting input. The non-inverting input of the fourth amplifier OA4 is connected to the positive terminal of the Schottky diode D3. The voltage input Vin is connected to resistor R3. The current output Iout is output from the negative terminal of the Schottky diode D3.

[0034] like Figure 4 As shown, the Hall bar array convolution module includes a Hall bar array composed of multiple Hall bars, an operational amplifier, and an analog multiply-accumulate circuit; the Hall bar array receives the current signal I output from the signal conversion module. NThe analog multiply-accumulate circuit converts the anomalous Hall resistance R of the Hall bar. hallbarN As the multiplicand, the operational amplifier gain A, and the current signal as the multiplier, all anomalous Hall voltages (I) after dot multiplication are... N *R hallbarN *A) Perform addition operations and output the result of the convolution operation.

[0035] like Figure 5 As shown, when the Hall bar is driven by a pulsed current to reverse its magnetic moment, the resistance of the anomalous Hall resistor changes accordingly, and the resistance is non-volatile after the control ends. The ratio of the anomalous Hall voltage generated at the output terminal of the Hall bar to the input current is such that, due to this characteristic, the anomalous Hall voltage output by the Hall bar itself can achieve a multiplication operation between the anomalous Hall resistance and the current.

[0036] The activation pooling storage array is used to implement ReLU activation and max pooling storage. The result of the convolution operation is used as input, and the pooling voltage is stored in the storage array module. The storage array module is the storage circuit of the array gating storage module. The capacitor C is used to store the voltage. The storage voltage in the storage array is input to the next layer Hall bar array convolution activation pooling storage module through the action of the controller. After passing through multiple series-connected signal conversion modules, Hall bar array convolution modules, and activation pooling storage array modules, the result is directly output in the form of analog quantity.

[0037] The controller acts as a scheduling module for each stage. Convolutional kernels are obtained through training, and the controller writes them into the Hall bar array, resulting in the anomalous Hall resistance values ​​of the Hall bars.

[0038] like Figure 6 As shown, the implementation principle of this invention is as follows:

[0039] s1, such as Figure 4 As shown, the controller uses a write operation to write the trained convolutional kernel into the Hall bar array, and the corresponding result is the anomalous Hall resistance value of the Hall bar;

[0040] s2. After modifying the anomalous Hall resistance value of the Hall bar array, the controller turns on the field-effect transistor M1 in the voltage storage hardware circuit of the controllable selection path through the control signal Vs, completes the discharge of the storage capacitor C, and controls the initialization signal of the convolution activation pooling to prepare for the next operation.

[0041] s3. After initialization is complete, the controller controls the gating circuit to store the sensing voltages of the corresponding rows and columns of the sensing array and input them into the signal conversion module.

[0042] s4. After receiving the sensing voltage from the sensor array, the signal conversion module converts it into the corresponding Hall bar input current;

[0043] s5. After receiving the input current from the signal conversion module, the Hall bar array convolution module performs dot multiplication and summation of the multiple input currents and the anomalous Hall resistance of the Hall bar corresponding to the convolution kernel through an analog multiply-accumulate circuit to obtain the convolution result, which is then input into the activation pooling storage array module for storage.

[0044] s6. After receiving the convolution result from the Hall bar array convolution module, the activated pooling storage array module stores the convolution result in capacitor C.

[0045] s7. The controller determines whether the pooling operation storage is complete. If the pooling is not completed, it returns to s3 and performs convolution calculation on the next set of data. If the controller determines that the pooling operation storage is complete, it will store the voltage input signal in the pooling array in the voltage input signal conversion module.

[0046] s8. After receiving the input voltage, the signal conversion module converts it into the corresponding Hall bar input current;

[0047] s9. After receiving the input current from the signal conversion module, the Hall bar array convolution module performs dot multiplication and summation of the multiple input currents and the anomalous Hall resistance of the Hall bar corresponding to the convolution kernel through an analog multiply-accumulate circuit to obtain the convolution result, which is then input into the activation pooling storage array module for storage.

[0048] s10. After receiving the convolution result from the Hall bar array convolution module, the activated pooling storage array stores the convolution result in capacitor C. The ellipsis indicates that more layers of signal conversion modules, Hall bar array convolution modules, and activated pooling storage array modules are added as needed.

[0049] s11. The controller determines whether the calculation is finished. If not, it returns to s2, initializes the first layer Hall bar array convolution activation pooling storage module, and then performs convolution pooling calculation for the next set of data. If finished, it directly outputs the analog quantity, and the Hall bar-based sensing, storage, and computing integrated system completes one round of calculation.

Claims

1. A multi-layer convolutional pooling in-memory computing system based on Hall bar arrays, characterized in that, It includes a sensor array, an array gating and storage module, a signal conversion module, a Hall bar array convolution module, an activation pooling and storage array module, and a controller; The sensing array, as a data acquisition module for analog signals, receives different sensing signals and outputs voltage signals; the array gating and storage module stores the voltage signals through a voltage storage hardware circuit with a controllable gating path and outputs the voltage value corresponding to the convolution kernel in the convolution window. The signal conversion module receives voltage values ​​and converts them into current signals for the corresponding Hall bars of the convolution kernel. The Hall bar array convolution module includes a Hall bar array composed of multiple Hall bars, an operational amplifier, and an analog multiply-accumulate circuit, outputting the result of the convolution operation. The activation pooling storage array is used to implement ReLU activation and max pooling storage, taking the result of the convolution operation as input, storing the pooling voltage in the storage array module, and outputting the result in the form of an analog quantity. The controller acts as a scheduling module, obtaining the convolution kernel through training, and writing it into the Hall bar array using the controller. The array gating storage module also includes an operational amplifier module, which is used to amplify the voltage output by the sensor array by a factor before inputting it into the storage voltage circuit. The voltage storage hardware circuit with a controllable selection path is used to realize the data storage of selected rows and columns. It includes a first amplifier OA1, diodes D1 and D2, a second amplifier OA2, resistors R1 and R2, a capacitor C, and a field-effect transistor M1. The output terminal of the first amplifier OA1 is connected to the anode of diode D1. The cathode of diode D1 is connected to the non-inverting input terminal of the second amplifier OA2. The anode of diode D2 is connected to the inverting input terminal of the first amplifier OA1. The cathode of diode D2 is connected to the anode of diode D1. One end of resistor R2 is connected to the inverting input terminal of the first amplifier OA1, and the other end is connected to the output terminal and the inverting input terminal of the second amplifier OA2. One end of resistor R1 is connected to the non-inverting input terminal of the first amplifier OA1, and the other end is grounded. The two ends of capacitor C are connected to the source of field-effect transistor M1 and the non-inverting input terminal of the second amplifier OA2, respectively. The drain of field-effect transistor M1 is connected to the non-inverting input terminal of the second amplifier OA2, the gate is connected to the control signal output by the controller, and the source is grounded.

2. The multi-layer convolutional pooling sensing-in-memory computing system based on Hall strip array according to claim 1, characterized in that, The amplitude of the current signal output by the signal conversion module does not exceed the critical current value that changes the Hall resistance.

3. The multi-layer convolutional pooling sensing-in-memory computing system based on Hall strip array according to claim 2, characterized in that, The signal conversion module is used to convert voltage signals into corresponding current signals. It includes a third amplifier OA3, resistors R3, R4, R5, R6, and R7, a Schottky diode D3, and a fourth amplifier OA4. The output of the third amplifier OA3 is connected to one end of resistor R5; the other end of resistor R5 is connected to the non-inverting input of the fourth amplifier OA4; one end of resistor R4 is connected to the inverting input of the third amplifier OA3, and the other end is connected to the output of the third amplifier OA3; the two ends of resistor R6 are connected to the inverting input of the third amplifier OA3 and ground, respectively; the output of the fourth amplifier OA4 is connected to one end of resistor R7; the other end of resistor R7 is connected to the non-inverting input of the third amplifier OA3; the output of the fourth amplifier OA4 is connected to the inverting input of the fourth amplifier OA4; the non-inverting input of the fourth amplifier OA4 is connected to the positive terminal of the Schottky diode D3; the voltage input Vin is connected to resistor R3; and the current output Iout is output from the negative terminal of the Schottky diode D3.

4. The multi-layer convolutional pooling sensing-in-memory computing system based on Hall strip array according to claim 1, characterized in that, The Hall bar array receives the current signal I output by the signal conversion module. N The analog multiply-accumulate circuit converts the anomalous Hall resistance R of the Hall bar. hallbarN The operational amplifier gain A is used as the multiplicand, and the current signal is used as the multiplier. All anomalous Hall voltages after dot product are added together, and the result of the convolution operation is output.

5. The multi-layer convolutional pooling sensing-in-memory computing system based on Hall strip array according to claim 1, characterized in that, The Hall bar's magnetic moment flips under pulse current drive, causing the anomalous Hall resistor value to change accordingly. After the control ends, the resistance value is non-volatile. The ratio between the anomalous Hall voltage generated at the output terminal of the Hall bar and the input current is such that the anomalous Hall voltage output by the Hall bar itself realizes the multiplication operation between the anomalous Hall resistor and the current.

6. The multi-layer convolutional pooling sensing-in-memory computing system based on Hall strip arrays according to any one of claims 1 to 5, characterized in that, The storage array module is the storage circuit of the array gating storage module. The capacitor C is used to store the voltage. The storage voltage in the storage array is input to the next layer Hall bar array convolution activation pooling storage module through the controller. After passing through multiple series-connected signal conversion modules, Hall bar array convolution modules, and activation pooling storage array modules, the result is output in the form of an analog quantity.

Images