Flexible intelligent sensing electronic skin based on FPC

Abstract

The invention relates to an FPC-based flexible intelligent sensing electronic skin, belongs to the technical field of integrated circuits, and solves the problem of poor contact sensing stability in the fields of robots, medical health and the like in the prior art. The skin comprises a flexible touch sensor array, a row selection module and a sensor interface circuit. The flexible touch sensor array is used for sensing an external pressure signal and converting the pressure signal into a current signal; the row selection module is used for selecting rows of the flexible touch sensor array through high and low levels; and the sensor interface circuit is used for selecting columns of the flexible tactile sensor array and converting the current signals of the array elements corresponding to the selected rows and columns into digital signals which are used for representing the pressure value of the flexible tactile sensor array. The characteristics of softness, stability and safety of the flexible sensing electronic skin are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated circuits, and in particular to a flexible intelligent sensing electronic skin based on FPC. Background Art

[0002] With the development of the Internet of Things and artificial intelligence, intelligent devices and systems have experienced explosive growth. Among them, intelligent devices in vision and imaging have experienced many leaps and bounds. However, due to the limitations of application scenarios and conditions, such as in dark scenes and the perception of the material of objects, under such conditions, relying solely on visual perception cannot meet special applications. However, after losing vision, the human body can still rely on touch to complete many complex perceptions and operations. The human brain has a specific level in processing visual and somatosensory information. When the human body cannot receive visual information, it can still complete the perception and feedback of external stimuli through tactile information.

[0003] In addition, the research on tactile perception systems has also had extremely important significance in the field of medical health. For example, doctors usually touch the patient's body with their hands and fingers to feel its softness to assess the exact location of pain and locate specific anatomical structures and abnormalities during surgery. Digitizing the process of touch-based recognition of the human body can provide quantitative and objective information to the diagnostician, which will change the traditional clinically subjective and experience-based palpation and bring new changes to the field of medical health.

[0004] In actual applications, electronic skin needs to be attached to uneven places such as human hands or robotic arms that need to perform various actions. This requires electronic skin to have good electrical connection stability while ensuring extremely high flexibility. Therefore, in order to solve the contact perception problem that restricts the further development of robotics, medical health and other fields, it is urgent to design and manufacture a soft, stable and safe electronic skin. Summary of the invention

[0005] In view of the above analysis, an embodiment of the present invention aims to provide a flexible intelligent sensing electronic skin based on FPC to solve the problem of poor contact perception stability in the fields of robotics, medical health, etc. in the prior art.

[0006] On the one hand, an embodiment of the present invention provides a flexible intelligent sensing electronic skin based on FPC, the skin comprising a flexible tactile sensor array, a row selection module and a sensor interface circuit;

[0007] The flexible tactile sensor array is used to sense external pressure signals and convert the pressure signals into current signals;

[0008] The row selection module selects the rows of the flexible tactile sensor array through high and low levels;

[0009] The sensor interface circuit selects columns of the flexible tactile sensor array and converts current signals of array elements corresponding to the selected rows and columns into digital signals for representing pressure values ​​of the flexible tactile sensor array.

[0010] Furthermore, the flexible tactile sensor array is a thin film pressure sensor array, and the thin film pressure sensor array is integrated on a flexible FPC board.

[0011] Furthermore, the size of the tactile sensor array is 7 rows and 5 columns, and each array element is provided with a thin film pressure sensor.

[0012] Further, the sensor interface circuit includes: an operational amplifier and an analog-to-digital converter ADC;

[0013] The operational amplifier converts the current signal into a voltage signal and transmits the voltage signal to the analog-to-digital converter ADC;

[0014] The analog-to-digital converter ADC converts the voltage signal into a digital signal.

[0015] Furthermore, the thin film pressure sensor includes a positive electrode and a negative electrode. The positive electrodes of the flexible tactile sensor array are connected in horizontal rows, and the negative electrodes are connected in vertical columns. Each thin film pressure sensor is a tactile pixel, and the pressure information of each pixel is read by row scanning.

[0016] Furthermore, the connection relationship between the operational amplifier, the analog-to-digital converter ADC and the row selection module is:

[0017] The row selection module includes an analog switch, which acts as a single-pole double-throw switch. One end of each row of the flexible tactile sensor array is connected to a single-pole double-throw switch and the moving end of the single-pole double-throw switch. The first fixed end of the single-pole double-throw is grounded, and the second fixed end is connected to the non-inverting input end of the operational amplifier. The other end of each row of the flexible tactile sensor array is connected to the inverting input end of the operational amplifier. The output end of the operational amplifier is connected to one end of the feedback resistor and the input end of the analog-to-digital converter ADC. The other end of the feedback resistor is connected to the inverting input end of the operational amplifier. The output end of the analog-to-digital converter ADC is grounded, and the non-inverting input end of the operational amplifier is grounded through a reference voltage.

[0018] Furthermore, the voltage detection formula of the flexible intelligent sensing electronic skin is:

[0019]

[0020] Among them, V get is the actual voltage value of the thin film pressure sensor, V ref is the reference voltage, R gis the feedback resistor, R x is the resistance of the thin film pressure sensor.

[0021] Furthermore, the FPC board is cut into the shape of a hand by laser, and the five thin film pressure sensors in the first row of the flexible tactile sensor array are respectively placed at the fingertips of the first joints of the five fingers, the first thin film pressure sensor in the second row is placed at the rear end of the first joint of the thumb, and the remaining four thin film pressure sensors are respectively placed at the second joints of the remaining four fingers, the first thin film pressure sensor in the third row is placed at the second joint of the thumb, and the remaining four thin film pressure sensors are respectively placed at the third joints of the remaining four fingers, and the thin film pressure sensors in the fourth to seventh rows are evenly distributed on the palm.

[0022] Furthermore, the thin film pressure sensor is cut into discs adapted to the size of the electrode by laser cutting, and the thin film pressure sensor is fixed on the hand-shaped FPC board by using acrylic tape.

[0023] On the other hand, an embodiment of the present invention provides a flexible tactile system that can identify the softness and hardness of an object. The flexible tactile system is used to obtain external pressure values ​​according to the above-mentioned FPC-based flexible intelligent sensing electronic skin, and identify the softness and hardness of the object based on the pressure value.

[0024] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0025] 1. The electronic flexible skin of the present application evenly arranges the flexible tactile sensor array on the FPC flexible board and cuts it into the shape of a hand, which can be applied to robots or prostheses. By reasonably arranging the positions of the thin film pressure sensors on the hands, the softness and hardness of the objects grasped in the hands can be accurately sensed.

[0026] 2. This application realizes the tactile perception of electronic skin through the combined use of flexible peripheral circuits and BNN computing chips. It does not require hardware connection to a PC like traditional tactile perception systems. It directly uses BNN computing chips to process sensor data. There is no need to upload it to the cloud or connect to the Internet, so there is no information security problem and the calculation speed is faster.

[0027] 3. This application uses a flexible RRAM storage and computing array with a 1T1R unit structure. The indium gallium zinc oxide thin-film transistor IGZO TFT and memristor in the 1T1R unit structure are both flexible devices. Compared with the traditional 2T2R structure, this application reduces the storage and computing array area by half without affecting the calculation accuracy, further improving energy efficiency. Its flexible characteristics are more suitable for creating electronic skin, and the entire system is flexible and more comfortable. It will also be more durable than a rigid system, and can be better integrated with the human body to realize wearable electronic products.

[0028] 4. Flexible peripheral circuits such as analog-to-digital converter ADC, popcount module, multiplier and subtractor can fully calculate the entire BNN algorithm and can input multiple bits, which improves the calculation precision and accuracy of the hardware circuit implementation of the BNN algorithm.

[0029] 5. In this application, a flexible tactile sensor array and a flexible peripheral circuit are used. The flexible peripheral circuit and the BNN computing chip are integrated on a flexible PCB board to create a thin, soft, highly stable, and highly integrated integrated flexible tactile system.

[0030] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0032] Figure 1 is a schematic diagram of a flexible tactile sensor array;

[0033] Figure 2 It is a schematic diagram for connecting analog-to-digital converter ADC, analog switch and column sensor;

[0034] Figure 3 Schematic diagram of the piezoresistive sensor electrode;

[0035] Figure 4 The design schematic diagram of FPC for this application;

[0036] Figure 5 Physical picture of the electronic skin for this application;

[0037] Figure 6 Circuit diagram of the flexible RRAM storage array

[0038] Figure 7 This is a diagram of the architecture of the flexible tactile system of this application;

[0039] Figure 8 The data flow chart of the BNN algorithm provided by the present invention. DETAILED DESCRIPTION

[0040] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0041] Example 1

[0042] A specific embodiment of the present invention discloses a flexible intelligent sensing electronic skin based on FPC, such as Figure 1 As shown. The skin includes a flexible tactile sensor array, a row selection module and a sensor interface circuit;

[0043] The flexible tactile sensor array is used to sense external pressure signals and convert the pressure signals into current signals;

[0044] The row selection module selects the rows of the flexible tactile sensor array through high and low levels;

[0045] The sensor interface circuit selects columns of the flexible tactile sensor array and converts current signals of array elements corresponding to the selected rows and columns into digital signals for representing pressure values ​​of the flexible tactile sensor array.

[0046] Furthermore, the flexible FPC board is cut into the shape of a hand by laser, and the five thin film pressure sensors in the first row of the flexible tactile sensor array are respectively placed at the fingertips of the first joints of the five fingers, the first thin film pressure sensor in the second row is placed at the rear end of the first joint of the thumb, and the remaining four thin film pressure sensors are respectively placed at the second joints of the remaining four fingers, the first thin film pressure sensor in the third row is placed at the second joint of the thumb, and the remaining four thin film pressure sensors are respectively placed at the third joints of the remaining four fingers, and the thin film pressure sensors in the fourth to seventh rows are evenly distributed on the palm.

[0047] Furthermore, the flexible tactile sensor array is a thin film pressure sensor array, and the thin film pressure sensor array is integrated on a flexible FPC board.

[0048] Specifically, the material of the flexible FPC is polyimide with a thickness of 0.13 mm. It is a wearable electronic skin device that makes it easy to stick the electronic skin to gloves. It also has a hollow design at the fold of the hand. The copper wire connecting the sensor array is embedded inside the flexible FPC to form an electrical connection for the entire circuit.

[0049] Furthermore, the size of the tactile sensor array is 7 rows and 5 columns, and each array element is provided with a thin film pressure sensor.

[0050] Furthermore, the thin film pressure sensor is cut into discs adapted to the size of the electrode by laser cutting, and the thin film pressure sensor is fixed on the hand-shaped FPC board by using acrylic tape.

[0051] Further, such as Figure 3 As shown, the thin film pressure sensor includes a positive electrode and a negative electrode. The positive electrodes of the flexible tactile sensor array are connected in horizontal rows, and the negative electrodes are connected in vertical columns. Each thin film pressure sensor is a tactile pixel, and the pressure information of each pixel is read by row scanning.

[0052] Specifically, each thin-film pressure sensor has a piezoresistive film pasted on the positive and negative electrodes to form a resistor. When the piezoresistive film is subjected to pressure changes, the resistance value of the resistor will change. The 35 resistors are connected in a row-column cross method. The row selection module selects any row to apply voltage, and each column can obtain an output voltage value. When the resistor receives external pressure, the output voltage changes, which reflects the size of the output voltage.

[0053] The flexible piezoresistive sensor film material is Velostat conductive copolymer with a thickness of 0.1 mm.

[0054] Specifically, the sensing principle of the thin film pressure sensor is a piezoresistive sensor. When force acts on the sensor, the resistance of the corresponding tactile pixel on the sensor changes. The greater the pressure, the smaller the resistance, and the smaller the sensing current, so that the pressure signal can be converted into a current signal for processing.

[0055] Furthermore, the voltage detection formula of the flexible intelligent sensing electronic skin is:

[0056]

[0057] Among them, V get is the actual voltage value of the thin film pressure sensor, V ref is the reference voltage, R g is the feedback resistor, R x is the resistance of the thin film pressure sensor.

[0058] Specifically, V get The value range is 1-5V, V ref =1V, R g =5.9KΩ, sensor R x The range is from 1.4KΩ to ∞. g To adjust the sensitivity of the acquisition circuit to slight signals, or to reduce the feedback resistance R g To shield the slight signal. At the same time, it can be seen that the voltage obtained is inversely proportional to the resistance to be measured.

[0059] Further, the sensor interface circuit includes: an operational amplifier and an analog-to-digital converter ADC;

[0060] The operational amplifier converts the current signal into a voltage signal and transmits the voltage signal to the analog-to-digital converter ADC;

[0061] The analog-to-digital converter ADC converts the voltage signal into a digital signal.

[0062] Further, such as Figure 2 As shown, the connection relationship between the operational amplifier, the analog-to-digital converter ADC and the row selection module is:

[0063] The row selection module includes an analog switch, which acts as a single-pole double-throw switch. One end of each row of the flexible tactile sensor array is connected to a single-pole double-throw switch and the moving end of the single-pole double-throw switch. The first fixed end of the single-pole double-throw is grounded, and the second fixed end is connected to the non-inverting input end of the operational amplifier. The other end of each row of the flexible tactile sensor array is connected to the inverting input end of the operational amplifier. The output end of the operational amplifier is connected to one end of the feedback resistor and the input end of the analog-to-digital converter ADC. The other end of the feedback resistor is connected to the inverting input end of the operational amplifier. The output end of the analog-to-digital converter ADC is grounded, and the non-inverting input end of the operational amplifier is grounded through a reference voltage.

[0064] Specifically, the tactile sensor has 7 rows and 5 columns, each column is connected to an operational amplifier, and each operational amplifier is connected to a row selection module.

[0065] Specifically, when the resistance value of the first row and the first column is selected for measurement, according to the virtual short and virtual open characteristics of the operational amplifier, the voltage of the inverting input terminal of the amplifier is equal to the voltage of the non-inverting input terminal, which is set to 1V. The reference voltage of the first row is set to 0V through the row selection module, and the reference voltage of the remaining 6 rows is 1V, that is, the moving end of the first single-pole double-throw switch is connected to the first fixed end, and the moving ends of the remaining 6 single-pole double-throw switches are connected to the second fixed end, that is, the voltage across the first row of thin film pressure sensors is 1V, and the voltage across the remaining 6 rows of thin film pressure sensors is equal to 0V, and the current passing through the resistor is 0A. After the first row is selected, the operational amplifier is used to select the column to be measured. After each column of the first row is measured, the row selection module is used to select the next row, that is, each tactile sensor can be selected row by row through the coordinated use of the row selection module and the operational amplifier.

[0066] Example 2

[0067] On the other hand, an embodiment of the present invention provides a flexible tactile system that can identify the softness and hardness of an object. The flexible tactile system is used to obtain external pressure values ​​according to the FPC-based flexible intelligent sensing electronic skin described in Example 1, and identify the softness and hardness of the object based on the pressure values.

[0068] Specifically, Figure 7As shown, the flexible tactile system includes: a flexible tactile sensor array, a flexible peripheral circuit and a BNN computing chip;

[0069] The flexible tactile sensor array is the flexible electronic skin in Example 1.

[0070] The flexible tactile sensor array is used to sense external pressure signals and convert the pressure signals into current signals, and read the current signals of each row of the sensor array to the flexible peripheral circuit by row scanning;

[0071] The flexible peripheral circuit converts the current signal of each row into a digital signal, and performs amplification, denoising and filtering to obtain the pressure value of each row and transmit it to the BNN calculation chip;

[0072] The BNN calculation chip performs convolution operation on the pressure value to obtain a hardness classification result.

[0073] Furthermore, the flexible tactile sensor array is a thin film pressure sensor array, which is integrated on a flexible FPC board.

[0074] Further, such as Figure 1 As shown, the flexible tactile sensor array has a size of 7 rows and 5 columns, and each array element is provided with a thin film pressure sensor.

[0075] The sensing principle of the thin film pressure sensor is a piezoresistive sensor. When force acts on the sensor, the resistance of the corresponding tactile pixel on the sensor changes. The greater the pressure, the smaller the resistance, and the smaller the sensing current, so that the pressure signal can be converted into a current signal for processing.

[0076] Preferably, dense wiring in a glove shape is integrated on the FPC board, and the board is cut into a glove shape and pasted on the glove, which has advantages in terms of wearability and higher reliability of electrical connection.

[0077] Specifically, Figure 1 and 2 As shown, the connection relationship between the analog-to-digital converter ADC, the analog switch and the 7 thin film pressure sensors in each column, the analog switch is used to select the output row, the analog-to-digital converter ADC is used to select the output column, the pressure signals on the thin film pressure sensors in each column are selected row by row, the current signals on the 35 thin film pressure sensors in 7 rows and 5 columns are selected and input into the flexible peripheral circuit in turn, and stored in the buffer storage module in the peripheral circuit after processing, a total of 35 pressure values ​​are obtained, each pressure value has 8 bits, and these 35 pressure values ​​are input into the BNN calculation chip bit by bit for soft and hardness identification.

[0078] Further, the flexible peripheral circuit includes: a sensor interface circuit and a signal processing circuit;

[0079] The sensor interface circuit converts the current signal into a digital signal and transmits it to the signal processing circuit;

[0080] The signal processing circuit amplifies, removes noise and performs filtering on the digital signal.

[0081] Specifically, the flexible peripheral circuit also includes a power supply module and a buffer storage module. The power supply module is responsible for the power supply of the entire system, including powering the flexible tactile sensor array, allowing the sensor to generate a current signal to display the pressure signal, and providing the BNN computing chip with a power supply voltage and a bias voltage; the buffer storage module is used to store the digital signal after amplification, denoising and filtering output by the signal processing circuit, and transmit it to the BNN computing chip.

[0082] Furthermore, the sensor interface circuit includes an operational amplifier and an analog-to-digital converter ADC;

[0083] The operational amplifier converts the current signal into a voltage signal and transmits the voltage signal to the analog-to-digital converter ADC;

[0084] The analog-to-digital converter ADC converts the voltage signal into a digital signal.

[0085] Specifically, the main function of the signal processing circuit is to amplify, denoise and filter the input digital signal to obtain a noise-free digital signal, and store the 5 columns of signals transmitted each time into the buffer storage module. After seven times, the complete 35 signals of the flexible tactile sensor array are obtained and transmitted to the BNN computing chip together, so that the calculation results transmitted to the BNN computing chip are less disturbed;

[0086] Furthermore, the flexible peripheral circuit and the BNN computing chip are both integrated on a flexible PCB board.

[0087] Further, such as Figure 8 As shown, the BNN computing chip includes: a flexible RRAM storage array, a multiplication and accumulation module, a shifter, a comparator and a maximum value comparison output module;

[0088] According to the BNN network structure and input data, two areas are selected in the flexible RRAM storage array as a first algorithm network and a second algorithm network;

[0089] Input multiple rows of data of each bit into the first algorithm network and the multiplication-accumulation module, the first algorithm network outputs the accumulated current of each column of the storage and calculation unit to the multiplication-accumulation module, the multiplication-accumulation module obtains the multiplication-accumulation result of each column based on the multiple rows of data and the accumulated current of each column, and inputs the multiplication-accumulation result of each column to the shifter;

[0090] The shifter shifts the multiplication and accumulation results of each column by the corresponding number of bits according to the bit positions of each row of input data, and outputs the shifted result to the adder-subtractor in the multiplication and accumulation module. The adder-subtractor performs sum calculation on the shifted results of different bit positions of each column, and outputs the sum calculation result of each column to the comparator;

[0091] The comparator performs binarization processing on the sum calculation results of each column to obtain a binarization result, outputs it to the second algorithm network, calculates and outputs the accumulated current of each column again in the second algorithm network, and sends it to the maximum comparator output module;

[0092] The maximum comparator output module determines the maximum value of the accumulated current in each column, and the column where the maximum value is located is the classification result.

[0093] Specifically, the input data of the BNN computing chip is the pressure value output by the flexible tactile sensor array, which is processed by the sensor interface circuit and the signal processing circuit and then stored in the buffer storage module as a digital signal after amplification, denoising and filtering.

[0094] Specifically, according to the network structure of the BNN algorithm and the amount of input data, two areas are selected in the flexible RRAM storage array as the first algorithm network and the second algorithm network.

[0095] Specifically, the rows and columns of the first algorithm network are determined according to the number of rows of input data and the size of the hidden layer, and then the size of the first algorithm network is determined. The rows and columns of the second algorithm network are determined according to the size of the hidden layer and the number of output data of the output layer, and then the size of the second algorithm network is determined.

[0096] Specifically, the main function of the BNN computing chip is to realize neural network reasoning that can perform soft and hardness classification. The BNN computing circuit is implemented using low-temperature polycrystalline oxide technology, which is different from traditional CMOS circuits.

[0097] Furthermore, the flexible RRAM storage and computing array includes n×m storage and computing units;

[0098] Each of the storage and computing units has a 1T1R structure, including a transistor and a memristor, and the transistor is an indium gallium zinc oxide thin film transistor IGZO TFT.

[0099] Specifically, the transistors and memristors are both flexible devices; the word lines WL of each row of storage and calculation units are connected in sequence, the source lines SL of each column of storage and calculation units are connected in sequence, and the bit lines BL of each column of storage and calculation units are connected in sequence, and the bit lines BL are used to output the calculation results of the storage and calculation units in this column;

[0100] Preferably, Figure 6As shown, the flexible RRAM storage array has a size of 64*64 and a total capacity of 4Kb; it cooperates with flexible peripheral circuits such as word line decoders, bit line decoders, and source line decoders to form a multiplication and addition calculation unit. The word line decoder, bit line decoder, and source line decoder decode the address signal given by the external FPGA to cooperate with it to complete the read and write operations.

[0101] Specifically, the transistor is an indium gallium zinc oxide thin film transistor IGZO TFT, and the memristor is RRAM;

[0102] The gate of the indium gallium zinc oxide thin film transistor IGZO TFT is connected to the word line WL of the storage unit, the drain is connected to the bit line BL of the storage unit, the source is connected to one end of the memristor RRAM, and the other end of the RRAM is connected to the source line SL of the storage unit.

[0103] Specifically, the working modes of the flexible RRAM storage and computing array include a write weight mode and an inference working mode.

[0104] Specifically, when the bit line BL or the source line SL is connected to the write voltage Vw, the working mode of the flexible RRAM storage array is the write weight mode, and when the bit line BL is connected to the read voltage Vr, the working mode of the flexible RRAM storage array is the inference working mode.

[0105] Specifically, the input voltages of the word lines, bit lines, and source lines connected to the array are controlled through the peripheral FPGA, that is, the word line WL is connected to a high level or a low level to control the on and off of the transistor, thereby selecting whether the current storage unit is enabled; when the word line WL is connected to a high level, the transistor is turned on, that is, the storage unit is enabled; when the word line WL is connected to a low level, the transistor is not turned on, that is, the storage unit is not enabled.

[0106] Specifically, in the write weight working mode, the connection voltages of the word line WL, the bit line BL and the source line SL are controlled to change the resistance state of the memristor RRAM, thereby achieving weight writing.

[0107] Specifically, when the word line WL is connected to a high level, the bit line BL is connected to a write voltage Vw, and the source line SL is grounded, the conductance state of the memristor RRAM is a low resistance state LRS, and its weight is assigned to be β+1;

[0108] When the word line WL is connected to a high level, the bit line BL is grounded, and the source line SL is connected to a write voltage Vw, the conductance state of the memristor RRAM is a high resistance state HRS, and its weight is assigned to β-1;

[0109] When the word line WL is connected to a low level, it indicates that the memristor RRAM is not selected, and the resistance state of the memristor RRAM remains unchanged.

[0110] In the write weight mode, the high and low resistance states of the RRAM are set by controlling the write voltage Vw of the storage and calculation unit and the on and off of the transistor, thereby completing the deployment of the weight in the RRAM array; the output of the source line SL and the bit line BL is grounded or connected to the write voltage, and the word line WL controls the on and off of the transistor to control the weight writing of the RRAM in the storage and calculation unit. For example, when the word line WL <0> When the source line SL is connected to the ground terminal and the bit line BL is connected to the write voltage, the RRAM is programmed to a low resistance state.

[0111] Specifically, in the write weight mode and the inference working mode, because the memristor RRAM is a non-volatile device, after the write weight mode completes the resistance state setting of the RRAM, the resistance state of the memristor RRAM is fixed and will not change unless a new programming voltage is applied to the word line WL, the bit line BL and the source line SL.

[0112] Specifically, the write weight mode adopts a half voltage method, the write voltage Vw is greater than the threshold voltage of the memristor RRAM, and the half write voltage Vw is less than the threshold voltage of the memristor RRAM to ensure that the state of the unselected memristor RRAM remains unchanged.

[0113] Specifically, in the inference working mode, the bit line BL is connected to the read voltage Vr and the source line SL is grounded. The input of the storage unit is determined according to whether the bit line WL is connected to a high level or a low level. The current value output by the storage unit is taken as the product of the input of the storage unit and the weight of the memristor RRAM. The currents output by each storage unit in each column of storage units are combined in the bit line corresponding to the storage unit in the column, so that the bit line output is the sum of the products of each unit in the column of storage units.

[0114] Specifically, Figure 1 As shown in FIG. 1 , when the word line WL is connected to a high level and the resistance state of the memristor RRAM is a high resistance state HRS, the input of the storage unit is 1 and the output is a low current I L , low current I L The value of is β-1;

[0115] When the word line WL is connected to a high level and the resistance state of the memristor RRAM is a low resistance state LRS, the input of the storage unit is 1 and the output current I H , high current I H The value of is β+1;

[0116] When the word line WL is connected to a low level, the indium gallium zinc oxide thin film transistor IGZO TFT is disconnected, the storage unit is not connected, the input of the storage unit is 0, and the output current of the storage unit is 0.

[0117] In the inference working mode, the entire RRAM array can realize the calculation function of multiplication and addition; the on and off of the TFT transistor is controlled by the word line WL to realize the input: when the word line WL is connected to a high level, the TFT transistor is turned on, that is, the storage unit is selected, indicating that the input is 1; when the word line WL is connected to a low level, the transistor is turned off, that is, the unit is not connected, indicating that the input is 0. The source line SL is connected to the ground, and the bit line BL is connected to the read voltage Vr to obtain the corresponding accumulated current, which is the multiplication and addition result. RRAM represents the weight distribution through the high and low resistance states after programming. When a bit line BL of the RRAM array is selected, a current I will be generated according to different high resistance states or low resistance states. H or I L For example, when the word line WL <0> When the RRAM is in a high impedance state, the storage unit generates a reference current I L , which means that the calculation result of the unit is β-1 (β>1), and so on for the calculation of a whole column of storage and calculation units. The accumulated current obtained at the output bit line BL is the multiplication and addition result of a column of storage and calculation units. In this way, only one TFT transistor and one RRAM can be used as a storage and calculation unit to form a storage and calculation array.

[0118] Specifically, the β>1, the present application is equivalent to a deformed binary calculation, the input of the word line WL is to control whether the transistor is turned on or not, and the high resistance state and low resistance state of the memristor RRAM are determined by controlling the voltage connected to the bit line BL and the source line SL in the write weight mode. Since the high resistance state and the low resistance state are actually positive values, the low resistance state and the high resistance state are encoded as β+1 and β-1 respectively. In the inference working mode, the output also changes accordingly, and the output is also a binary high current and a low current.

[0119] Furthermore, the multiplication-accumulation module further includes: a popcount module and a multiplier;

[0120] The popcount module receives multiple lines of input and obtains the number of 1s in the multiple lines of input, and outputs the number of 1s to the multiplier;

[0121] The multiplier multiplies the number of 1s by a fixed value β and sends the product result to the adder-subtractor;

[0122] The adder-subtractor subtracts the product result from the digital signal to obtain the matrix multiplication and addition result, and sends it to the shifter.

[0123] Specifically, the function of the multiplier is to multiply the input by a fixed β value to solve the weight setting problem, where the β value is determined by the performance of the RRAM storage array.

[0124] Exemplarily, when the matrix multiplication is [1,0,1,0,1]*[+1,+1,-1,-1,-1], that is, multiple rows of input are [1,0,1,0,1], the weight of a column of the RRAM storage array is [+1,+1,-1,-1,-1], the multiplication and addition result is -1, and the corresponding multiple rows of input of the RRAM storage array are 10101. The resistance states of the input corresponding RRAM units are β+1, β+1, β-1, β-1, and β-1, respectively. Then, the accumulated current output by the RRAM storage array in this column is input to the multiplexer, and the multiplexer selects the accumulated current of the column of the storage array to be output to the analog-to-digital converter ADC. After analog-to-digital conversion, a current digital signal of 3β-1 is output to the adder-subtractor. The popcount module output is 3, and the multiplier output is 3β. Then the adder-subtractor performs a subtraction operation and outputs -1. It can be seen that this scheme is completely consistent with the original multiplication and addition result.

[0125] Specifically, the circuit further includes: a first column multiplexer MUX, a first analog-to-digital converter ADC;

[0126] The first column multiplexer MUX sequentially selects each column storage and calculation unit of the first algorithm network, and sends the accumulated current of the selected column storage and calculation unit to the first analog-to-digital converter ADC;

[0127] The first analog-to-digital converter ADC converts the accumulated current output by the first algorithm network into a digital signal and sends it to the adder-subtractor.

[0128] Specifically, the adder-subtractor implements addition operation or subtraction operation through the control signal sent by the FPGA.

[0129] Specifically, the adder-subtractor module adopts a carry-lookahead adder structure, which can perform addition and subtraction of two signed numbers. The addition or subtraction is controlled by the control signal given by the FPGA. Because the RRAM storage array weight encoding adopts all-positive number encoding during hardware implementation, it is actually greater than the weight value in the actual algorithm. Therefore, the multiplication and addition result from the RRAM array is subtracted from the output result of the popcount module to make a correction, and subtraction is performed at this time; this implementation method is to split the input into each bit input, so the results of each bit need to be added to get the calculation result of the complete input, and addition is performed at this time.

[0130] Furthermore, the BNN computing chip further includes: a first data buffer module, a second data buffer module and a third data buffer module; all of which are used for temporary storage and integration of data;

[0131] The first data buffer module receives the digital signal output by the first analog-to-digital converter ADC, buffers it, and then outputs it to the adder-subtractor;

[0132] The second data buffer module receives the shift result output by the shifter, buffers it, and then outputs it to the adder-subtractor;

[0133] The third data buffer module receives the binarization result output by the comparator, buffers it, and then outputs it to the second algorithm network.

[0134] Specifically, the circuit further includes: a second column multiplexer MUX, a second analog-to-digital converter ADC;

[0135] The second column multiplexer MUX sequentially selects each column storage and calculation unit of the second algorithm network, and sends the accumulated current of the selected column storage and calculation unit to the second analog-to-digital converter ADC;

[0136] The second analog-to-digital converter ADC converts the accumulated current output by the second algorithm network into a digital signal and sends it to the maximum value comparison output module.

[0137] Specifically, according to the bit positions of the input multiple rows of data, the multiplication and addition result is shifted to the left by the corresponding number of bits in the shifter; for the multiplication and accumulation result corresponding to the i-th multiple rows of data, the multiplication and accumulation result is shifted to the left by i bits in the shifter;

[0138] Among them, 0≤i≤7.

[0139] Specifically, the shifter is used to perform data shift of a signed number instead of a multiplication operation.

[0140] Furthermore, a threshold is preset in the comparator. When the sum calculation result is greater than the threshold, the comparator outputs 1, and when the sum calculation result is less than the threshold, the comparator outputs 0.

[0141] Specifically, the comparator implements normalization and binarization of the data for use by the next layer of the network.

[0142] The maximum value comparison output module can compare the maximum values ​​of the four output data of the last layer of the network and use them as the output of the classification result.

[0143] Exemplarily, the flexible tactile sensor array is an array of 5 rows and 7 columns, and the output pressure values ​​are 35, that is, the input data of the RRAM array in the BNN computing chip is 35 rows of 8-bit data. If the hidden layer size in the BNN network is 14, the BNN network output is 4 data, so the required algorithm network is two fully connected layers of 35*14 and 14*4, which are mapped to the circuit implementation as two RRAM arrays of 35*14 and 14*4. Therefore, according to the above BNN network structure and input data, a 35*14 area is selected in the flexible RRAM storage array as the first algorithm network. , select a 14*4 area as the second algorithm network; the first input of the flexible RRAM storage array is 35 rows*8bit input, and the weights are reused. Each bit of 35 is written into the first algorithm network through the word line decoder eight times, that is, the input data is each bit of multiple rows (35 rows), and the multiplication and addition result is obtained after calculation through the first algorithm network. The first column multiplexer MUX is connected to the first analog-to-digital converter ADC to obtain the digital-to-analog conversion result of the accumulated current of each column, and the analog-to-digital conversion result is stored in the data buffer module 1. The output of the data buffer module 1 is connected to the adder-subtractor. At the same time, when each bit of the 35 rows is input, it is connected to the popcount module, and the popcount module obtains the number of 1s in each bit of the multiple rows of input. The output of the popcount module is connected to the multiplier, and the number of 1s is multiplied by a fixed β value in the multiplier, and the output of the multiplier is connected to the adder-subtractor. The FPGA first sets the adder-subtractor to subtraction, and the output of the data buffer module 1 is subtracted from the output of the multiplier to obtain the multiplication and accumulation result of each bit of the 35 rows. The multiplication and accumulation results of each column corresponding to each bit are input into the shifter, and the corresponding shift of each bit is performed. For example, the 0th bit is shifted to the left by 0 bits, and the 7th bit is shifted to the left by 7 bits. The shift results of each bit of each column are stored in the data buffer module 2. For example, if it is the 0th bit, the input data of the shifter is 00000010, which is shifted by 0 bits, and the shift result is 00000010 after the shift; if it is the 3rd bit, the input data of the shifter is 00000010, which is shifted by 3 bits, and the shift result is 00010000. After the above operations are performed on all 8 bits, the shift results obtained are stored in the data buffer module 2, and the shift results in the data buffer module 2 are input into the adder-subtractor. The FPGA sets the adder-subtractor to addition, and all the bit shift results of each column stored in the data buffer module 2 are added to obtain the sum of each column, and the sum result is output to the comparator.A normalization operation is performed in the comparator, that is, the sum of the addition results is compared with the set threshold value. When the sum result is greater than the threshold value, 1 is output, and when the sum result is less than the threshold value, 0 is output; that is, when the input addition result is 3 and the threshold value is 1, the output is 1; when the input addition result is 0 and the threshold value is 1, the output is 0; in this way, the binary input of the next layer is obtained, that is, the input data of the second algorithm network; the last layer is not binarized, and the classification result is directly output through the maximum value comparison output module. Specifically, if the accumulated current value of the first column is the largest, 0 is output, and so on. If the data in the fourth column is the largest, 3 is output, corresponding to the four hardnesses of soft, softer, harder and hard respectively.

[0144] Specifically, the present invention provides a flexible tactile system that can identify the softness and hardness of an object. The system can integrate sensing and computing, perform computing and processing of tactile information, and obtain the softness and hardness information of the object. The use of edge intelligent BNN computing chips can reduce system power consumption and improve the speed and safety of the tactile system; at the same time, the sensors and circuits are integrated on a flexible substrate, making the entire system flexible like electronic skin, with the advantages of being wearable and easy to use.

[0145] Compared with the prior art, the flexible intelligent sensing electronic skin based on FPC provided in this embodiment evenly arranges the flexible tactile sensor array on the FPC flexible board and cuts it into the shape of a hand, which can be applied to robots or prostheses. By reasonably arranging the positions of the thin film pressure sensors on the hand, the softness and hardness of the object grasped in the hand can be accurately sensed.

[0146] This application realizes the tactile perception of electronic skin through the combined use of flexible peripheral circuits and BNN computing chips. It does not require hardware connection to a PC like traditional tactile perception systems. The BNN computing chip is used directly to process sensor data. There is no need to upload it to the cloud or connect to the Internet, so there is no information security problem and the speed will be faster.

[0147] This application uses a flexible RRAM storage and computing array with a 1T1R unit structure. The indium gallium zinc oxide thin-film transistor IGZO TFT and memristor in the 1T1R unit structure are both flexible devices. Compared with the traditional 2T2R structure, this application reduces the storage and computing array area by half without affecting the calculation accuracy, further improving energy efficiency. Its flexible characteristics are more suitable for creating electronic skin, and the entire system is flexible and more comfortable. It is also more durable than a rigid system and can be better integrated with the human body to realize wearable electronic products.

[0148] Flexible peripheral circuits such as analog-to-digital converters (ADCs), popcount modules, multipliers and subtractors can fully calculate the entire BNN algorithm and can input multiple bits, thereby improving the computational precision and accuracy of the hardware circuit implementation of the BNN algorithm.

[0149] In this application, a flexible tactile sensor array and a flexible peripheral circuit are used. The flexible peripheral circuit and the BNN computing chip are integrated on a flexible PCB board to create a thin, soft, highly stable, and highly integrated integrated flexible tactile system.

[0150] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0151] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A flexible intelligent sensing electronic skin based on FPC, characterized in that: The skin includes a flexible tactile sensor array, a row selection module, and a sensor interface circuit; The flexible tactile sensor array is used to sense external pressure signals and convert the pressure signals into current signals; The row selection module selects the rows of the flexible tactile sensor array through high and low levels; The sensor interface circuit selects columns of the flexible tactile sensor array and converts current signals of array elements corresponding to the selected rows and columns into digital signals for representing pressure values ​​of the flexible tactile sensor array.

2. The FPC-based flexible intelligent sensing electronic skin according to claim 1, characterized in that: The flexible tactile sensor array is a thin film pressure sensor array, and the thin film pressure sensor array is integrated on a flexible FPC board.

3. The FPC-based flexible intelligent sensing electronic skin according to claim 2, characterized in that: The size of the tactile sensor array is 7 rows and 5 columns, and each array element is provided with a thin film pressure sensor.

4. The FPC-based flexible intelligent sensing electronic skin according to claim 3, characterized in that: The sensor interface circuit includes: an operational amplifier and an analog-to-digital converter ADC; The operational amplifier converts the current signal into a voltage signal and transmits the voltage signal to the analog-to-digital converter ADC; The analog-to-digital converter ADC converts the voltage signal into a digital signal.

5. The FPC-based flexible intelligent sensing electronic skin according to claim 4, characterized in that: The thin film pressure sensor includes a positive electrode and a negative electrode. The positive electrodes of the flexible tactile sensor array are connected in horizontal rows, and the negative electrodes are connected in vertical columns. Each thin film pressure sensor is a tactile pixel, and the pressure information of each pixel is read by row scanning.

6. The FPC-based flexible intelligent sensing electronic skin according to claim 5, characterized in that: The connection relationship between the operational amplifier, the analog-to-digital converter ADC and the row selection module is: The row selection module includes an analog switch, which acts as a single-pole double-throw switch. One end of each row of the flexible tactile sensor array is connected to a single-pole double-throw switch and the moving end of the single-pole double-throw switch. The first fixed end of the single-pole double-throw is grounded, and the second fixed end is connected to the non-inverting input end of the operational amplifier. The other end of each row of the flexible tactile sensor array is connected to the inverting input end of the operational amplifier. The output end of the operational amplifier is connected to one end of the feedback resistor and the input end of the analog-to-digital converter ADC. The other end of the feedback resistor is connected to the inverting input end of the operational amplifier. The output end of the analog-to-digital converter ADC is grounded, and the non-inverting input end of the operational amplifier is grounded through a reference voltage.

7. The FPC-based flexible intelligent sensing electronic skin according to claim 6, characterized in that: The voltage detection formula of the flexible intelligent sensing electronic skin is: Among them, V get is the actual voltage value of the thin film pressure sensor, V ref is the reference voltage, R g is the feedback resistor, R x is the resistance of the thin film pressure sensor.

8. The FPC-based flexible intelligent sensing electronic skin according to claim 3, characterized in that: The FPC board is cut into the shape of a hand by laser. The five thin film pressure sensors in the first row of the flexible tactile sensor array are placed on the fingertips of the first joints of the five fingers, the first thin film pressure sensor in the second row is placed at the rear end of the first joint of the thumb, and the remaining four thin film pressure sensors are placed on the second joints of the remaining four fingers, the first thin film pressure sensor in the third row is placed on the second joint of the thumb, and the remaining four thin film pressure sensors are placed on the third joints of the remaining four fingers, and the thin film pressure sensors in the fourth to seventh rows are evenly distributed on the palm.

9. The FPC-based flexible intelligent sensing electronic skin according to claim 8, characterized in that: The thin film pressure sensor is cut into discs that adapt to the size of the electrode by laser cutting, and the thin film pressure sensor is fixed on the hand-shaped FPC board with acrylic tape.

10. A flexible tactile system capable of identifying the softness and hardness of an object, characterized in that: The flexible tactile system is used for obtaining external pressure values ​​according to the FPC-based flexible intelligent sensing electronic skin according to any one of claims 1 to 9, and identifying the softness and hardness of the object based on the pressure values.

Images