Flexible touch system capable of identifying hardness of object

Abstract

The invention relates to a flexible touch system capable of identifying the hardness of an object, belongs to the technical field of integrated circuits, and solves the problems that a rigid system in the prior art is easy to break, needs hardware to be connected with a PC (Personal Computer), processes data and uploads the data to a cloud end, cannot run without networking and is low in speed. The system comprises a flexible touch sensor array, a peripheral circuit and a BNN calculation chip, the flexible touch sensor array is used for sensing an external pressure signal, converting the pressure signal into a current signal, and reading the current signal of each row of the sensor array to the peripheral circuit in a row scanning mode; the peripheral circuit is used for converting the current signals of each row into digital signals, amplifying, denoising and filtering the digital signals to obtain pressure values of each row, and transmitting the pressure values to the BNN calculation chip; and the BNN calculation chip is used for carrying out BNN algorithm operation on the pressure value to obtain a hardness classification result. On the premise of keeping the original network precision, the circuit area is reduced, the energy consumption is reduced, the energy efficiency of the whole storage circuit is improved, and the safety is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated circuits, and in particular to a flexible tactile system capable of identifying the softness and hardness of an object. Background Art

[0002] Human cognition of the environment and objects is the result of the combined action of multiple senses. Most previous research was based on vision and hearing, which are relatively mature, while tactile perception is still in the exploratory stage. Electronic skin is actually a new type of sensor that imitates the perception function of human skin. It is soft, flexible and wearable. Like human skin, it can also sense external stimuli and convert these stimuli into different output signals. Simply put, electronic skin can give robots a "sense of touch", can be processed into various shapes, and can be attached to the surface of the device like clothes. Therefore, electronic skin can play a big role in smart medical care, human-computer interaction (HMI), virtual reality (VR) and artificial intelligence (AI).

[0003] However, traditional electronic skins also have many disadvantages. First, most electronic skins need to be integrated with external devices such as PCs, data collectors, etc., and require multi-device coordination, which is very inconvenient. Secondly, except for the sensing part, most components are rigid, which greatly affects the beauty, comfort and safety of the electronic skin, and also has an adverse effect on signal acquisition. Finally, most of the current electronic skins need to upload data to the cloud or computer and then perform data analysis on the computing device, which has disadvantages in speed and security.

[0004] Based on the above inconveniences, we urgently need to develop and build an integrated flexible tactile system that is light, soft, highly stable, and highly integrated with computing chips. Summary of the invention

[0005] In view of the above analysis, an embodiment of the present invention aims to provide a flexible tactile system that can identify the softness and hardness of an object, so as to solve the problems in the prior art that the rigid system is easy to break, requires hardware to be connected to a PC, data processing needs to be uploaded to the cloud, and cannot run without an Internet connection and has a slow speed.

[0006] On the one hand, an embodiment of the present invention provides a flexible tactile system capable of identifying the softness and hardness of an object, the system comprising: a flexible tactile sensor array, a flexible peripheral circuit and a BNN computing chip;

[0007] The flexible tactile sensor array is used to sense external pressure signals and convert the pressure signals into current signals, and read the current signals of each row of the sensor array to the flexible peripheral circuit by row scanning;

[0008] The flexible peripheral circuit converts the current signal of each row into a digital signal, and performs amplification, denoising and filtering to obtain the pressure value of each row and transmit it to the BNN calculation chip;

[0009] The BNN calculation chip performs convolution operation on the pressure value to obtain a hardness classification result.

[0010] Furthermore, the flexible peripheral circuit and the BNN computing chip are both integrated on a flexible PCB board.

[0011] Furthermore, the BNN computing chip includes: a flexible RRAM storage array, a multiplication and accumulation module, a shifter, a comparator, and a maximum value comparison output module;

[0012] According to the BNN network structure and input data, two areas are selected in the flexible RRAM storage array as a first algorithm network and a second algorithm network;

[0013] Input multiple rows of data of each bit into the first algorithm network and the multiplication-accumulation module, the first algorithm network outputs the accumulated current of each column of the storage and calculation unit to the multiplication-accumulation module, the multiplication-accumulation module obtains the multiplication-accumulation result of each column based on the multiple rows of data and the accumulated current of each column, and inputs the multiplication-accumulation result of each column to the shifter;

[0014] The shifter shifts the multiplication and accumulation results of each column by the corresponding number of bits according to the bit positions of each row of input data, and outputs the shifted result to the adder-subtractor in the multiplication and accumulation module. The adder-subtractor performs sum calculation on the shifted results of different bit positions of each column, and outputs the sum calculation result of each column to the comparator;

[0015] The comparator performs binarization processing on the sum calculation results of each column to obtain a binarization result, outputs it to the second algorithm network, calculates and outputs the accumulated current of each column again in the second algorithm network, and sends it to the maximum comparator output module;

[0016] The maximum comparator output module determines the maximum value of the accumulated current in each column, and the column where the maximum value is located is the classification result.

[0017] Furthermore, the flexible RRAM storage and computing array includes n×m storage and computing units;

[0018] Each of the storage and computing units has a 1T1R structure, including a transistor and a memristor, and the transistor is an indium gallium zinc oxide thin film transistor IGZO TFT.

[0019] Furthermore, the flexible tactile sensor array is a thin film pressure sensor array, and the thin film pressure sensor array is integrated on a flexible FPC board.

[0020] Furthermore, the size of the flexible tactile sensor array is 7 rows and 5 columns, and each array element is provided with a thin film pressure sensor.

[0021] Further, the flexible peripheral circuit includes: a sensor interface circuit, a signal processing circuit;

[0022] The sensor interface circuit converts the current signal into a digital signal and transmits it to the signal processing circuit;

[0023] The signal processing circuit amplifies, removes noise and performs filtering on the digital signal.

[0024] Furthermore, the sensor interface circuit includes an operational amplifier and an analog-to-digital converter ADC;

[0025] The operational amplifier converts the current signal into a voltage signal and transmits the voltage signal to the analog-to-digital converter ADC;

[0026] The analog-to-digital converter ADC converts the voltage signal into a digital signal.

[0027] Furthermore, the multiplication-accumulation module further includes: a popcount module and a multiplier;

[0028] The popcount module receives multiple lines of input and obtains the number of 1s in the multiple lines of input, and outputs the number of 1s to the multiplier;

[0029] The multiplier multiplies the number of 1s by a fixed value β and sends the product result to the adder-subtractor;

[0030] The adder-subtractor subtracts the product result from the digital signal to obtain the matrix multiplication and addition result, and sends it to the shifter.

[0031] Furthermore, the BNN computing chip further includes: a first data buffer module, a second data buffer module and a third data buffer module; all of which are used for temporary storage and integration of data;

[0032] The first data buffer module receives the digital signal output by the first analog-to-digital converter ADC, buffers it, and then outputs it to the adder-subtractor;

[0033] The second data buffer module receives the shift result output by the shifter, buffers it, and then outputs it to the adder-subtractor;

[0034] The third data buffer module receives the binarization result output by the comparator, buffers it, and then outputs it to the second algorithm network.

[0035] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0036] 1. This application realizes the tactile perception of electronic skin through the combined use of flexible peripheral circuits and BNN computing chips. It does not require hardware connection to a PC like traditional tactile perception systems. The BNN computing chip is used directly to process sensor data. There is no need to upload it to the cloud or connect to the Internet, so there is no information security problem and the speed will be faster.

[0037] 2. This application uses a flexible RRAM storage and computing array with a 1T1R unit structure. The indium gallium zinc oxide thin-film transistor IGZO TFT and memristor in the 1T1R unit structure are both flexible devices. Compared with the traditional 2T2R structure, this application reduces the storage and computing array area by half without affecting the calculation accuracy, further improving energy efficiency. Its flexible characteristics are more suitable for creating electronic skin, and the entire system is flexible, more comfortable, and more durable than a rigid system.

[0038] 3. Flexible peripheral circuits such as analog-to-digital converter ADC, popcount module, multiplier and subtractor can fully calculate the entire BNN algorithm and can input multiple bits, which improves the calculation precision and accuracy of the hardware circuit implementation of the BNN algorithm.

[0039] 4. In this application, a flexible tactile sensor array and a flexible peripheral circuit are used. The flexible peripheral circuit and the BNN computing chip are integrated on a flexible PCB board to create a thin, soft, highly stable, and highly integrated integrated flexible tactile system.

[0040] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0042] Figure 1 This is a diagram of the architecture of the flexible tactile system of this application;

[0043] Figure 2 A data flow chart for implementing the BNN algorithm provided by the present invention;

[0044] Figure 3 This is a circuit diagram of a flexible RRAM storage and computing array;

[0045] Figure 4Schematic diagram of a flexible tactile sensor array;

[0046] Figure 5 Schematic diagram for connecting analog-to-digital converter ADC, analog switch and column sensor. DETAILED DESCRIPTION

[0047] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0048] A specific embodiment of the present invention discloses a flexible tactile system capable of identifying the softness and hardness of an object. Figure 1 As shown. The system includes: a flexible tactile sensor array, a flexible peripheral circuit and a BNN computing chip;

[0049] The flexible tactile sensor array is used to sense external pressure signals and convert the pressure signals into current signals, and read the current signals of each row of the sensor array to the flexible peripheral circuit by row scanning;

[0050] The flexible peripheral circuit converts the current signal of each row into a digital signal, and performs amplification, denoising and filtering to obtain the pressure value of each row and transmit it to the BNN calculation chip;

[0051] The BNN calculation chip performs convolution operation on the pressure value to obtain a hardness classification result.

[0052] Furthermore, the flexible tactile sensor array is a thin film pressure sensor array, which is integrated on a flexible FPC board.

[0053] Further, such as Figure 4 As shown, the flexible tactile sensor array has a size of 7 rows and 5 columns, and each array element is provided with a thin film pressure sensor.

[0054] The sensing principle of the thin film pressure sensor is a piezoresistive sensor. When force acts on the sensor, the resistance of the corresponding tactile pixel on the sensor changes. The greater the pressure, the smaller the resistance, and the smaller the sensing current, so that the pressure signal can be converted into a current signal for processing.

[0055] Preferably, dense wiring in a glove shape is integrated on the FPC board, and the board is cut into a glove shape and pasted on the glove, which has advantages in terms of wearability and higher reliability of electrical connection.

[0056] Specifically, Figure 4 and 5As shown, the connection relationship between the analog-to-digital converter ADC, the analog switch and the 7 thin film pressure sensors in each column, the analog switch is used to select the output row, the analog-to-digital converter ADC is used to select the output column, the pressure signals on the thin film pressure sensors in each column are selected row by row, the current signals on the 35 thin film pressure sensors in 7 rows and 5 columns are selected and input into the flexible peripheral circuit in turn, and stored in the buffer storage module in the peripheral circuit after processing, a total of 35 pressure values ​​are obtained, each pressure value has 8 bits, and these 35 pressure values ​​are input into the BNN calculation chip bit by bit for soft and hardness identification.

[0057] Further, the flexible peripheral circuit includes: a sensor interface circuit and a signal processing circuit;

[0058] The sensor interface circuit converts the current signal into a digital signal and transmits it to the signal processing circuit;

[0059] The signal processing circuit amplifies, removes noise and performs filtering on the digital signal.

[0060] Specifically, the flexible peripheral circuit also includes a power supply module and a buffer storage module. The power supply module is responsible for the power supply of the entire system, including powering the flexible tactile sensor array, allowing the sensor to generate a current signal to display the pressure signal, and providing the BNN computing chip with a power supply voltage and a bias voltage; the buffer storage module is used to store the digital signal after amplification, denoising and filtering output by the signal processing circuit, and transmit it to the BNN computing chip.

[0061] Furthermore, the sensor interface circuit includes an operational amplifier and an analog-to-digital converter ADC;

[0062] The operational amplifier converts the current signal into a voltage signal and transmits the voltage signal to the analog-to-digital converter ADC;

[0063] The analog-to-digital converter ADC converts the voltage signal into a digital signal.

[0064] Specifically, the main function of the signal processing circuit is to amplify, denoise and filter the input digital signal to obtain a noise-free digital signal, and store the 5 columns of signals transmitted each time into the buffer storage module. After seven times, the complete 35 signals of the flexible tactile sensor array are obtained and transmitted to the BNN computing chip together, so that the calculation results transmitted to the BNN computing chip are less disturbed;

[0065] Furthermore, the flexible peripheral circuit and the BNN computing chip are both integrated on a flexible PCB board.

[0066] Further, such as Figure 2As shown, the BNN computing chip includes: a flexible RRAM storage array, a multiplication and accumulation module, a shifter, a comparator and a maximum value comparison output module;

[0067] According to the BNN network structure and input data, two areas are selected in the flexible RRAM storage array as a first algorithm network and a second algorithm network;

[0068] Input multiple rows of data of each bit into the first algorithm network and the multiplication-accumulation module, the first algorithm network outputs the accumulated current of each column of the storage and calculation unit to the multiplication-accumulation module, the multiplication-accumulation module obtains the multiplication-accumulation result of each column based on the multiple rows of data and the accumulated current of each column, and inputs the multiplication-accumulation result of each column to the shifter;

[0069] The shifter shifts the multiplication and accumulation results of each column by the corresponding number of bits according to the bit positions of each row of input data, and outputs the shifted result to the adder-subtractor in the multiplication and accumulation module. The adder-subtractor performs sum calculation on the shifted results of different bit positions of each column, and outputs the sum calculation result of each column to the comparator;

[0070] The comparator performs binarization processing on the sum calculation results of each column to obtain a binarization result, outputs it to the second algorithm network, calculates and outputs the accumulated current of each column again in the second algorithm network, and sends it to the maximum comparator output module;

[0071] The maximum comparator output module determines the maximum value of the accumulated current in each column, and the column where the maximum value is located is the classification result.

[0072] Specifically, the input data of the BNN computing chip is the pressure value output by the flexible tactile sensor array, which is processed by the sensor interface circuit and the signal processing circuit and then stored in the buffer storage module as a digital signal after amplification, denoising and filtering.

[0073] Specifically, according to the network structure of the BNN algorithm and the amount of input data, two areas are selected in the flexible RRAM storage array as the first algorithm network and the second algorithm network.

[0074] Specifically, the rows and columns of the first algorithm network are determined according to the number of rows of input data and the size of the hidden layer, and then the size of the first algorithm network is determined. The rows and columns of the second algorithm network are determined according to the size of the hidden layer and the number of output data of the output layer, and then the size of the second algorithm network is determined.

[0075] Specifically, the main function of the BNN computing chip is to realize neural network reasoning that can perform soft and hardness classification. The BNN computing circuit is implemented using low-temperature polycrystalline oxide technology, which is different from traditional CMOS circuits.

[0076] Furthermore, the flexible RRAM storage and computing array includes n×m storage and computing units;

[0077] Each of the storage and computing units has a 1T1R structure, including a transistor and a memristor, and the transistor is an indium gallium zinc oxide thin film transistor IGZO TFT.

[0078] Specifically, the transistors and memristors are both flexible devices; the word lines WL of each row of storage and calculation units are connected in sequence, the source lines SL of each column of storage and calculation units are connected in sequence, and the bit lines BL of each column of storage and calculation units are connected in sequence, and the bit lines BL are used to output the calculation results of the storage and calculation units in this column;

[0079] Preferably, Figure 3 As shown, the flexible RRAM storage array has a size of 64*64 and a total capacity of 4Kb; it cooperates with flexible peripheral circuits such as word line decoders, bit line decoders, and source line decoders to form a multiplication and addition calculation unit. The word line decoder, bit line decoder, and source line decoder decode the address signal given by the external FPGA to cooperate with it to complete the read and write operations.

[0080] Specifically, the transistor is an indium gallium zinc oxide thin film transistor IGZO TFT, and the memristor is RRAM;

[0081] The gate of the indium gallium zinc oxide thin film transistor IGZO TFT is connected to the word line WL of the storage unit, the drain is connected to the bit line BL of the storage unit, the source is connected to one end of the memristor RRAM, and the other end of the RRAM is connected to the source line SL of the storage unit.

[0082] Specifically, the working modes of the flexible RRAM storage and computing array include a write weight mode and an inference working mode.

[0083] Specifically, when the bit line BL or the source line SL is connected to the write voltage Vw, the working mode of the flexible RRAM storage array is the write weight mode, and when the bit line BL is connected to the read voltage Vr, the working mode of the flexible RRAM storage array is the inference working mode.

[0084] Specifically, the input voltages of the word lines, bit lines, and source lines connected to the array are controlled through the peripheral FPGA, that is, the word line WL is connected to a high level or a low level to control the on and off of the transistor, thereby selecting whether the current storage unit is enabled; when the word line WL is connected to a high level, the transistor is turned on, that is, the storage unit is enabled; when the word line WL is connected to a low level, the transistor is not turned on, that is, the storage unit is not enabled.

[0085] Specifically, in the write weight working mode, the connection voltages of the word line WL, the bit line BL and the source line SL are controlled to change the resistance state of the memristor RRAM, thereby achieving weight writing.

[0086] Specifically, when the word line WL is connected to a high level, the bit line BL is connected to a write voltage Vw, and the source line SL is grounded, the conductance state of the memristor RRAM is a low resistance state LRS, and its weight is assigned to be β+1;

[0087] When the word line WL is connected to a high level, the bit line BL is grounded, and the source line SL is connected to a write voltage Vw, the conductance state of the memristor RRAM is a high resistance state HRS, and its weight is assigned to β-1;

[0088] When the word line WL is connected to a low level, it indicates that the memristor RRAM is not selected, and the resistance state of the memristor RRAM remains unchanged.

[0089] In the write weight mode, the high and low resistance states of the RRAM are set by controlling the write voltage Vw of the storage and calculation unit and the on and off of the transistor, thereby completing the deployment of the weight in the RRAM array; the output of the source line SL and the bit line BL is grounded or connected to the write voltage, and the word line WL controls the on and off of the transistor to control the weight writing of the RRAM in the storage and calculation unit. For example, when the word line WL <0> When the source line SL is connected to the ground terminal and the bit line BL is connected to the write voltage, the RRAM is programmed to a low resistance state.

[0090] Specifically, in the write weight mode and the inference working mode, because the memristor RRAM is a non-volatile device, after the write weight mode completes the resistance state setting of the RRAM, the resistance state of the memristor RRAM is fixed and will not change unless a new programming voltage is applied to the word line WL, the bit line BL and the source line SL.

[0091] Specifically, the write weight mode adopts a half voltage method, the write voltage Vw is greater than the threshold voltage of the memristor RRAM, and the half write voltage Vw is less than the threshold voltage of the memristor RRAM to ensure that the state of the unselected memristor RRAM remains unchanged.

[0092] Specifically, in the inference working mode, the bit line BL is connected to the read voltage Vr and the source line SL is grounded. The input of the storage unit is determined according to whether the bit line WL is connected to a high level or a low level. The current value output by the storage unit is taken as the product of the input of the storage unit and the weight of the memristor RRAM. The currents output by each storage unit in each column of storage units are combined in the bit line corresponding to the storage unit in the column, so that the bit line output is the sum of the products of each unit in the column of storage units.

[0093] Specifically, Figure 4 As shown in FIG. 1 , when the word line WL is connected to a high level and the resistance state of the memristor RRAM is a high resistance state HRS, the input of the storage unit is 1 and the output is a low current I L , low current I L The value of is β-1;

[0094] When the word line WL is connected to a high level and the resistance state of the memristor RRAM is a low resistance state LRS, the input of the storage unit is 1 and the output current I H , high current I H The value of is β+1;

[0095] When the word line WL is connected to a low level, the indium gallium zinc oxide thin film transistor IGZO TFT is disconnected, the storage unit is not connected, the input of the storage unit is 0, and the output current of the storage unit is 0.

[0096] In the inference working mode, the entire RRAM array can realize the calculation function of multiplication and addition; the on and off of the TFT transistor is controlled by the word line WL to realize the input: when the word line WL is connected to a high level, the TFT transistor is turned on, that is, the storage unit is selected, indicating that the input is 1; when the word line WL is connected to a low level, the transistor is turned off, that is, the unit is not connected, indicating that the input is 0. The source line SL is connected to the ground, and the bit line BL is connected to the read voltage Vr to obtain the corresponding accumulated current, which is the multiplication and addition result. RRAM represents the weight distribution through the high and low resistance states after programming. When a bit line BL of the RRAM array is selected, a current I will be generated according to different high resistance states or low resistance states. H or I L For example, when the word line WL <0> When the RRAM is in a high impedance state, the storage unit generates a reference current I L , which means that the calculation result of the unit is β-1 (β>1), and so on for the calculation of a whole column of storage and calculation units. The accumulated current obtained at the output bit line BL is the multiplication and addition result of a column of storage and calculation units. In this way, only one TFT transistor and one RRAM can be used as a storage and calculation unit to form a storage and calculation array.

[0097] Specifically, the β>1, the present application is equivalent to a deformed binary calculation, the input of the word line WL is to control whether the transistor is turned on or not, and the high resistance state and low resistance state of the memristor RRAM are determined by controlling the voltage connected to the bit line BL and the source line SL in the write weight mode. Since the high resistance state and the low resistance state are actually positive values, the low resistance state and the high resistance state are encoded as β+1 and β-1 respectively. In the inference working mode, the output also changes accordingly, and the output is also a binary high current and a low current.

[0098] Furthermore, the multiplication-accumulation module further includes: a popcount module and a multiplier;

[0099] The popcount module receives multiple lines of input and obtains the number of 1s in the multiple lines of input, and outputs the number of 1s to the multiplier;

[0100] The multiplier multiplies the number of 1s by a fixed value β and sends the product result to the adder-subtractor;

[0101] The adder-subtractor subtracts the product result from the digital signal to obtain the matrix multiplication and addition result, and sends it to the shifter.

[0102] Specifically, the function of the multiplier is to multiply the input by a fixed β value to solve the weight setting problem, where the β value is determined by the performance of the RRAM storage array.

[0103] Exemplarily, when the matrix multiplication is [1,0,1,0,1]*[+1,+1,-1,-1,-1], that is, multiple rows of input are [1,0,1,0,1], the weight of a column of the RRAM storage array is [+1,+1,-1,-1,-1], the multiplication and addition result is -1, and the corresponding multiple rows of input of the RRAM storage array are 10101. The resistance states of the input corresponding RRAM units are β+1, β+1, β-1, β-1, and β-1, respectively. Then, the accumulated current output by the RRAM storage array in this column is input to the multiplexer, and the multiplexer selects the accumulated current of the column of the storage array to be output to the analog-to-digital converter ADC. After analog-to-digital conversion, a current digital signal of 3β-1 is output to the adder-subtractor. The popcount module output is 3, and the multiplier output is 3β. Then the adder-subtractor performs a subtraction operation and outputs -1. It can be seen that this scheme is completely consistent with the original multiplication and addition result.

[0104] Specifically, the circuit further includes: a first column multiplexer MUX, a first analog-to-digital converter ADC;

[0105] The first column multiplexer MUX sequentially selects each column storage and calculation unit of the first algorithm network, and sends the accumulated current of the selected column storage and calculation unit to the first analog-to-digital converter ADC;

[0106] The first analog-to-digital converter ADC converts the accumulated current output by the first algorithm network into a digital signal and sends it to the adder-subtractor.

[0107] Specifically, the adder-subtractor implements addition operation or subtraction operation through the control signal sent by the FPGA.

[0108] Specifically, the adder-subtractor module adopts a carry-lookahead adder structure, which can perform addition and subtraction of two signed numbers. The addition or subtraction is controlled by the control signal given by the FPGA. Because the RRAM storage array weight encoding adopts all-positive number encoding during hardware implementation, it is actually greater than the weight value in the actual algorithm. Therefore, the multiplication and addition result from the RRAM array is subtracted from the output result of the popcount module to make a correction, and subtraction is performed at this time; this implementation method is to split the input into each bit input, so the results of each bit need to be added to get the calculation result of the complete input, and addition is performed at this time.

[0109] Furthermore, the BNN computing chip further includes: a first data buffer module, a second data buffer module and a third data buffer module; all of which are used for temporary storage and integration of data;

[0110] The first data buffer module receives the digital signal output by the first analog-to-digital converter ADC, buffers it, and then outputs it to the adder-subtractor;

[0111] The second data buffer module receives the shift result output by the shifter, buffers it, and then outputs it to the adder-subtractor;

[0112] The third data buffer module receives the binarization result output by the comparator, buffers it, and then outputs it to the second algorithm network.

[0113] Specifically, the circuit further includes: a second column multiplexer MUX, a second analog-to-digital converter ADC;

[0114] The second column multiplexer MUX sequentially selects each column storage and calculation unit of the second algorithm network, and sends the accumulated current of the selected column storage and calculation unit to the second analog-to-digital converter ADC;

[0115] The second analog-to-digital converter ADC converts the accumulated current output by the second algorithm network into a digital signal and sends it to the maximum value comparison output module.

[0116] Specifically, according to the bit positions of the input multiple rows of data, the multiplication and addition result is shifted to the left by the corresponding number of bits in the shifter; for the multiplication and accumulation result corresponding to the i-th multiple rows of data, the multiplication and accumulation result is shifted to the left by i bits in the shifter;

[0117] Among them, 0≤i≤7.

[0118] Specifically, the shifter is used to perform data shift of a signed number instead of a multiplication operation.

[0119] Furthermore, a threshold is preset in the comparator. When the sum calculation result is greater than the threshold, the comparator outputs 1, and when the sum calculation result is less than the threshold, the comparator outputs 0.

[0120] Specifically, the comparator implements normalization and binarization of the data for use by the next layer of the network.

[0121] The maximum value comparison output module can compare the maximum values ​​of the four output data of the last layer of the network and use them as the output of the classification result.

[0122] Exemplarily, the flexible tactile sensor array is an array of 5 rows and 7 columns, and the output pressure values ​​are 35, that is, the input data of the RRAM array in the BNN computing chip is 35 rows of 8-bit data. If the hidden layer size in the BNN network is 14, the BNN network output is 4 data, so the required algorithm network is two fully connected layers of 35*14 and 14*4, which are mapped to the circuit implementation as two RRAM arrays of 35*14 and 14*4. Therefore, according to the above BNN network structure and input data, a 35*14 area is selected in the flexible RRAM storage array as the first algorithm network. , select a 14*4 area as the second algorithm network; the first input of the flexible RRAM storage array is 35 rows*8bit input, and the weights are reused. Each bit of 35 is written into the first algorithm network through the word line decoder eight times, that is, the input data is each bit of multiple rows (35 rows), and the multiplication and addition result is obtained after calculation through the first algorithm network. The first column multiplexer MUX is connected to the first analog-to-digital converter ADC to obtain the digital-to-analog conversion result of the accumulated current of each column, and the analog-to-digital conversion result is stored in the data buffer module 1. The output of the data buffer module 1 is connected to the adder-subtractor. At the same time, when each bit of the 35 rows is input, it is connected to the popcount module, and the popcount module obtains the number of 1s in each bit of the multiple rows of input. The output of the popcount module is connected to the multiplier, and the number of 1s is multiplied by a fixed β value in the multiplier, and the output of the multiplier is connected to the adder-subtractor. The FPGA first sets the adder-subtractor to subtraction, and the output of the data buffer module 1 is subtracted from the output of the multiplier to obtain the multiplication and accumulation result of each bit of the 35 rows. The multiplication and accumulation results of each column corresponding to each bit are input into the shifter, and the corresponding shift of each bit is performed. For example, the 0th bit is shifted to the left by 0 bits, and the 7th bit is shifted to the left by 7 bits. The shift results of each bit of each column are stored in the data buffer module 2. For example, if it is the 0th bit, the input data of the shifter is 00000010, which is shifted by 0 bits, and the shift result is 00000010 after the shift; if it is the 3rd bit, the input data of the shifter is 00000010, which is shifted by 3 bits, and the shift result is 00010000. After the above operations are performed on all 8 bits, the shift results obtained are stored in the data buffer module 2, and the shift results in the data buffer module 2 are input into the adder-subtractor. The FPGA sets the adder-subtractor to addition, and all the bit shift results of each column stored in the data buffer module 2 are added to obtain the sum of each column, and the sum result is output to the comparator.A normalization operation is performed in the comparator, that is, the sum of the addition results is compared with the set threshold value. When the sum result is greater than the threshold value, 1 is output, and when the sum result is less than the threshold value, 0 is output; that is, when the input addition result is 3 and the threshold value is 1, the output is 1; when the input addition result is 0 and the threshold value is 1, the output is 0; in this way, the binary input of the next layer is obtained, that is, the input data of the second algorithm network; the last layer is not binarized, and the classification result is directly output through the maximum value comparison output module. Specifically, if the accumulated current value of the first column is the largest, 0 is output, and so on. If the data in the fourth column is the largest, 3 is output, corresponding to the four hardnesses of soft, softer, harder and hard respectively.

[0123] Specifically, the present invention provides a flexible tactile system that can identify the softness and hardness of an object. The system can integrate sensing and computing, perform computing and processing of tactile information, and obtain the softness and hardness information of the object. The use of edge intelligent BNN computing chips can reduce system power consumption and improve the speed and safety of the tactile system; at the same time, the sensors and circuits are integrated on a flexible substrate, making the entire system flexible like electronic skin, with the advantages of being wearable and easy to use.

[0124] Compared with the prior art, this embodiment provides a flexible tactile system that can identify the softness and hardness of an object. Through the joint use of flexible peripheral circuits and BNN computing chips, the tactile perception of electronic skin is realized. It does not require hardware connection to a PC like traditional tactile perception systems. The BNN computing chip is used directly to process the sensor data. There is no need to upload it to the cloud or connect to the Internet, so there is no information security problem and the speed will be faster.

[0125] This application uses a flexible RRAM storage and computing array with a 1T1R unit structure. The indium gallium zinc oxide thin-film transistor IGZO TFT and memristor in the 1T1R unit structure are both flexible devices. Compared with the traditional 2T2R structure, this application reduces the storage and computing array area by half without affecting the calculation accuracy, further improving energy efficiency. Its flexible characteristics are more suitable for creating electronic skin, and the entire system is flexible, more comfortable, and more durable than a rigid system.

[0126] Flexible peripheral circuits such as analog-to-digital converters (ADCs), popcount modules, multipliers and subtractors can fully calculate the entire BNN algorithm and can input multiple bits, thereby improving the computational precision and accuracy of the hardware circuit implementation of the BNN algorithm.

[0127] Flexible tactile sensors and flexible peripheral circuits are used in this application. The flexible peripheral circuits and BNN computing chips are integrated on a flexible PCB board to create a thin, soft, highly stable, and highly integrated integrated flexible tactile system.

[0128] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0129] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A flexible tactile system capable of identifying the softness and hardness of an object, characterized in that: The system includes: a flexible tactile sensor array, a flexible peripheral circuit and a BNN computing chip; The flexible tactile sensor array is used to sense external pressure signals and convert the pressure signals into current signals, and read the current signals of each row of the sensor array to the flexible peripheral circuit by row scanning; The flexible peripheral circuit converts the current signal of each row into a digital signal, and performs amplification, denoising and filtering to obtain the pressure value of each row and transmit it to the BNN calculation chip; The BNN calculation chip performs convolution operation on the pressure value to obtain a hardness classification result.

2. A flexible tactile system capable of identifying the softness and hardness of an object according to claim 1, characterized in that: The flexible peripheral circuit and the BNN computing chip are both integrated on a flexible PCB board.

3. The flexible tactile system capable of identifying the softness and hardness of an object according to claim 1, characterized in that: The BNN computing chip includes: a flexible RRAM storage array, a multiplication and accumulation module, a shifter, a comparator and a maximum value comparison output module; According to the BNN network structure and input data, two areas are selected in the flexible RRAM storage array as a first algorithm network and a second algorithm network; Input multiple rows of data of each bit into the first algorithm network and the multiplication-accumulation module, the first algorithm network outputs the accumulated current of each column of the storage and calculation unit to the multiplication-accumulation module, the multiplication-accumulation module obtains the multiplication-accumulation result of each column based on the multiple rows of data and the accumulated current of each column, and inputs the multiplication-accumulation result of each column to the shifter; The shifter shifts the multiplication and accumulation results of each column by the corresponding number of bits according to the bit positions of each row of input data, and outputs the shifted result to the adder-subtractor in the multiplication and accumulation module. The adder-subtractor performs sum calculation on the shifted results of different bit positions of each column, and outputs the sum calculation result of each column to the comparator; The comparator performs binarization processing on the sum calculation results of each column to obtain a binarization result, outputs it to the second algorithm network, calculates and outputs the accumulated current of each column again in the second algorithm network, and sends it to the maximum comparator output module; The maximum comparator output module determines the maximum value of the accumulated current in each column, and the column where the maximum value is located is the classification result.

4. The flexible tactile system capable of identifying the softness and hardness of an object according to claim 3, characterized in that: The flexible RRAM storage and computing array includes n×m storage and computing units; Each of the storage and computing units has a 1T1R structure, including a transistor and a memristor, and the transistor is an indium gallium zinc oxide thin film transistor IGZO TFT.

5. The flexible tactile system capable of identifying the softness and hardness of an object according to claim 1, characterized in that: The flexible tactile sensor array is a thin film pressure sensor array, and the thin film pressure sensor array is integrated on a flexible FPC board.

6. The flexible tactile system capable of identifying the softness and hardness of an object according to claim 1, characterized in that: The flexible tactile sensor array has 7 rows and 5 columns, and each array element is provided with a thin film pressure sensor.

7. The flexible tactile system capable of identifying the softness and hardness of an object according to claim 2, characterized in that: The flexible peripheral circuit includes: a sensor interface circuit and a signal processing circuit; The sensor interface circuit converts the current signal into a digital signal and transmits it to the signal processing circuit; The signal processing circuit amplifies, removes noise and performs filtering on the digital signal.

8. The flexible tactile system capable of identifying the softness and hardness of an object according to claim 7, characterized in that: The sensor interface circuit includes an operational amplifier and an analog-to-digital converter ADC; The operational amplifier converts the current signal into a voltage signal and transmits the voltage signal to the analog-to-digital converter ADC; The analog-to-digital converter ADC converts the voltage signal into a digital signal.

9. The flexible tactile system capable of identifying the softness and hardness of an object according to claim 8, characterized in that: The multiplication-accumulation module further comprises: a popcount module and a multiplier; The popcount module receives multiple lines of input and obtains the number of 1s in the multiple lines of input, and outputs the number of 1s to the multiplier; The multiplier multiplies the number of 1s by a fixed value β and sends the product result to the adder-subtractor; The adder-subtractor subtracts the product result from the digital signal to obtain the matrix multiplication and addition result, and sends it to the shifter.

10. The flexible tactile system capable of identifying the softness and hardness of an object according to claim 9, characterized in that: The BNN computing chip also includes: a first data buffer module, a second data buffer module and a third data buffer module; all of which are used for temporary storage and integration of data; The first data buffer module receives the digital signal output by the first analog-to-digital converter ADC, buffers it, and then outputs it to the adder-subtractor; The second data buffer module receives the shift result output by the shifter, buffers it, and then outputs it to the adder-subtractor; The third data buffer module receives the binarization result output by the comparator, buffers it, and then outputs it to the second algorithm network.

Images