Image recognition device and method

Abstract

According to the image recognition device and method, a current resistance relational expression is obtained according to the resistance values of a plurality of second memristors, the resistance value of a second column selection switch and a plurality of test currents, and the resistance value when the column selection switch is conducted is considered; after the weight voltage obtained according to the conductance value of the first memristor and the current resistance relational expression is input to the row signal end of the memristor sub-array, accurate mapping of the conductance value of the first memristor and the weight of the neural network can be realized. The plurality of column currents output from the plurality of column signal ends of the memristor sub-array are multiplication accumulation of the to-be-identified voltage and the conductance value of the memristor sub-array on the column, so that the accuracy of the image identification result determined according to the plurality of column currents is very high; switch control and data control are separated to different FPGAs, so that the reliability, flexibility and isolation of the device can be improved; and the serial-parallel conversion module is arranged, so that the IO interface pressure of the FPGA can be reduced, and the expandability is improved.

Description

Technical Field

[0001] The present application belongs to the technical field of memristor arrays, and in particular, relates to an image recognition device and method. Background Art

[0002] Image recognition technology refers to the technology of using computers to process, analyze and understand images in order to identify targets and objects of various different patterns. Image recognition is usually implemented based on neural networks.

[0003] The advantage of the memristor array is that it combines storage and computing. By mapping the weights of the neural network to the conductance values ​​of the memristors in the memristor array, the pixel values ​​of the image to be identified can be converted into voltage values ​​input to the row signal terminals of the memristor array. The magnitude of the column current obtained from each column signal terminal of the memristor array represents the corresponding recognition result, so that the memristor array can realize the hardware recognition of the neural network for the image. The accuracy of the conductance value of the memristor will seriously affect the accuracy of image recognition. Therefore, how to achieve accurate mapping of the conductance value of the memristor and the weights of the neural network is a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0004] The purpose of the present application is to provide an image recognition device and method; the image recognition device and method provided by the present application can realize the accurate mapping of the conductance value of the memristor and the weight value of the neural network, so that the accuracy of image recognition is very high.

[0005] The technical solutions provided by this application are as follows: An image recognition device, comprising a control processing module, a gate selection switch, a row selection switch, a first column selection switch and a memristor array, wherein the memristor array comprises a plurality of first memristors arranged in an array; The control processing module is electrically connected to the input end of the gate selection switch, the input end of the row selection switch and the output end of the first column selection switch respectively; The output end of the gate selection switch is electrically connected to the gate signal end of the memristor array; The output end of the row selection switch is electrically connected to the row signal end of the memristor array; An input terminal of the first column selection switch is electrically connected to a column signal terminal of the memristor array; The control processing module is used to obtain the resistance values ​​of multiple second memristors with different resistance values ​​and the resistance values ​​of the second column selection switches, input a test voltage to the row signal terminal of each of the second memristors, receive a test current corresponding to the resistance value of each of the second memristors through the second column selection switch, and obtain a current-resistance relationship according to the resistance values ​​of the multiple second memristors, the resistance values ​​of the second column selection switches and the multiple test currents, wherein the multiple second memristors are memristors with known parameters, the second column selection switches are column selection switches with known parameters, the control processing module is electrically connected to the row signal terminal of the second memristor and the output terminal of the second column selection switch, respectively, and the column signal terminal of the second memristor is electrically connected to the input terminal of the second column selection switch; The control processing module is also used to convert the pixel value of the image to be identified to obtain the voltage to be identified. According to the image to be recognized, determining row and column information of a memristor subarray to be operated in the memristor array, and obtaining a weight voltage corresponding to the conductance value of each first memristor in the memristor subarray according to the conductance value of each first memristor and the current-resistance relationship, wherein the conductance value of each first memristor is mapped to a weight matrix of a neural network determined according to the image to be recognized; The control processing module is also used to control the closure of the gate selection switch, the row selection switch and the switch in the first column selection switch based on the row and column information of the memristor subarray, input the first gate voltage to the gate signal terminal of the memristor subarray, input each of the weight voltages to the row signal terminal of the memristor subarray accordingly, so that the weights in the weight matrix of the neural network are written into the memristor subarray, input the voltage to be identified into each row signal terminal of the memristor subarray, receive each column current output from each column signal terminal of the memristor subarray through each first column selection subswitches in the first column selection switch, and determine the recognition result of the image to be identified based on each of the column currents.

[0006] Optionally, The control processing module is further used to read the row and column information to be modulated in the memristor array in the weight modulation instruction, each target resistance value and each pulse voltage corresponding to the row and column information to be modulated, select the first memristor to be modulated currently from the memristor array based on the row and column information to be modulated, and control the gate selection switch, the row selection switch and the first column selection switch The switch corresponding to the first memristor to be modulated currently is closed; The control processing module is also used to input the second gate voltage to the gate signal end of the currently modulated first memristor, input the pulse voltage corresponding to the row and column information of the currently modulated first memristor to the row signal end of the currently modulated first memristor, receive the modulated current after the current output from the column signal end of the currently modulated first memristor flows through the first column selection sub-switch, obtain the modulation resistance value according to the relationship between the modulation current and the current-resistance formula, judge whether the modulation resistance value is equal to the target resistance value, and if not, adjust the pulse voltage so that the adjusted modulation resistance value is equal to the target resistance value, until the modulation resistance value of each of the first memristors corresponding to the row and column information to be modulated is equal to the corresponding target resistance value.

[0007] Optionally, the control processing module includes a host computer, a first FPGA unit and a second FPGA module; The host computer is electrically connected to the first end of the first FPGA unit and the first end of the second FPGA module respectively; The second end of the first FPGA unit is electrically connected to the second end of the second FPGA module; The third terminal of the first FPGA unit is electrically connected to the control terminal of the gate selection switch; The fourth terminal of the first FPGA unit is electrically connected to the control terminal of the row selection switch; The fifth terminal of the first FPGA unit is electrically connected to the control terminal of the first column selection switch; The third terminal of the second FPGA module is electrically connected to the input terminal of the gate selection switch; The fourth terminal of the second FPGA module is electrically connected to the input terminal of the row selection switch; A fifth terminal of the second FPGA module is electrically connected to an output terminal of the first column selection switch; The output end of the gate selection switch is electrically connected to the gate signal end of the memristor array; The output end of the row selection switch is electrically connected to the row signal end of the memristor array; An input terminal of the first column selection switch is electrically connected to a column signal terminal of the memristor array; The host computer is used to convert the pixel values ​​of the image to be identified to obtain the voltage to be identified, determine the row and column information of the memristor subarray to be operated in the memristor array according to the image to be identified, obtain the weight voltage corresponding to the conductance value of each first memristor in the memristor subarray and the current-resistance relationship according to the conductance value of each first memristor in the memristor subarray and the current-resistance relationship, generate an image recognition instruction according to the voltage to be identified, the row and column information of the memristor subarray and the weight voltage of each first memristor, and send it to the first FPGA unit and the second FPGA module, wherein the conductance value of each first memristor is mapped to the weight matrix of the neural network determined according to the image to be identified; The first FPGA unit is used to receive the image recognition instruction, read the row and column information of the memristor subarray in the image recognition instruction, select the row and column information of the currently operated first memristor from the row and column information of the memristor subarray, send the row and column information of the currently operated first memristor to the second FPGA module, send a first gate selection control signal to the gate selection switch, send a first row selection control signal to the row selection switch, and send a first column selection control signal to the first column selection switch; The gate selection switch is used to close a gate selection sub-switch in the gate selection switch according to the first gate selection control signal; The row selection switch is used to close a row selection sub-switch in the row selection switch according to the first row selection control signal; The first column selection switch is used to close a first column selection sub-switch in the first column selection switch according to the first column selection control signal; The second FPGA module is used to receive the image recognition instruction, read the voltage to be recognized, the row and column information of the memristor subarray and the weight voltage of each of the first memristors in the image recognition instruction, input the first gate voltage to the gate signal terminal of the currently operating first memristor, and input the weight voltage corresponding to the row and column information of the currently operating first memristor to the row signal terminal of the currently operating first memristor; The first FPGA unit is further configured to send a second gate selection control signal to the gate selection switch, send a second row selection control signal to the row selection switch, and send a second column selection control signal to the first column selection switch based on the row and column information of the memristor subarray after the corresponding weight voltage is written to the row signal terminal of each first memristor of the memristor subarray; The gate selection switch is further used to close a plurality of gate selection sub-switches in the gate selection switch according to the second gate selection control signal; The row selection switch is further used to close a plurality of row selection sub-switches in the row selection sub-switches according to the second row selection control signal; The first column selection switch is further used to close a plurality of column selection switches in the first column selection switch according to the second column selection control signal; The second FPGA module is also used to input the first gate voltage into the gate signal terminal of the memristor subarray, input the voltage to be identified into each row signal terminal of the memristor subarray, receive each column current output from each column signal terminal of the memristor subarray through each first column selection sub-switches in the first column selection switch, determine the recognition result of the image to be identified based on each column current, and send the recognition result to the host computer.

[0008] Optionally, The host computer is further used to send a weight modulation instruction, wherein the weight modulation instruction includes the row and column information to be modulated in the memristor array, each target resistance value corresponding to the row and column information to be modulated, and each pulse voltage; The first FPGA unit is used to receive the weight modulation instruction, read the row and column information to be modulated in the weight modulation instruction, select the currently modulated first memristor from the memristor array based on the row and column information to be modulated, and send the corresponding second gate selection control signal to the gate selection switch, send the second row selection control signal to the row selection switch, and send the second column selection control signal to the first column selection switch; The gate selection switch is used to close a gate selection sub-switch in the gate selection switch according to the second gate selection control signal; The row selection switch is used to close a row selection sub-switch in the row selection switch according to the second row selection control signal; The first column selection switch is used to close a first column selection sub-switch in the first column selection switch according to the second column selection control signal; The second FPGA module is used to receive the weight modulation instruction, read the row and column information to be modulated, each target resistance value and each pulse voltage corresponding to the row and column information to be modulated in the weight modulation instruction, input the second gate voltage to the gate signal end of the currently modulated first memristor, input the pulse voltage corresponding to the row and column information of the currently modulated first memristor to the row signal end of the currently modulated first memristor, receive the modulation current after the current output from the column signal end of the currently modulated first memristor flows through the first column selection sub-switch, and send the modulation current to the host computer; The host computer is further used to obtain a modulation resistance value according to the modulation current and the current-resistance relationship, and to determine whether the modulation resistance value is equal to the target resistance value, and if not, to send a voltage control signal to the second FPGA module; The second FPGA module is used to receive the voltage control signal, and based on the voltage control signal, input the adjusted pulse voltage to the row signal end of the currently modulated first memristor, so that the adjusted modulation resistance is equal to the target resistance, until the modulation resistance of each of the first memristors corresponding to the row and column information to be modulated is equal to the corresponding target resistance, wherein the adjusted modulation resistance is obtained based on the adjusted modulation current output from the column signal end of the currently modulated first memristor and the current-resistance relationship.

[0009] Optionally, it further includes a first serial-to-parallel conversion module, the first serial-to-parallel conversion module includes a shift register and a register, the shift register includes N first triggers, the register includes N second triggers, and the gate selection switch is provided with N, wherein N is a positive integer greater than 5; The data input terminal of the shift register is electrically connected to the first sub-port in the third terminal of the first FPGA unit; The clock input terminal of each of the first triggers in the shift register is electrically connected to the second sub-port in the third terminal of the first FPGA unit; The reset input terminal of each of the first triggers in the shift register is electrically connected to the third sub-port in the third terminal of the first FPGA unit; The data output terminals of the first flip-flops in the shift register are electrically connected to the data input terminals of the second flip-flops in the register in a one-to-one correspondence; The clock input terminal of each of the second triggers in the register is electrically connected to the fourth subport in the third terminal of the first FPGA unit; The reset input terminal of each of the second flip-flops in the register is electrically connected to the fifth subport in the third terminal of the first FPGA unit; The data output end of each of the second triggers in the register is electrically connected to the control end of the N gate selection switches in a one-to-one correspondence; The first FPGA unit is used to convert an N-bit parallel gate selection control signal into an N-bit serial gate selection control signal and input it into the data input end of the shift register, and then input N first clock pulse signals to the clock input end of each of the first triggers in the shift register, so that the N-bit serial gate selection control signal is shifted and registered in the N second triggers in the register, and after the serial gate selection control signal is shifted and registered in the register, send a second time pulse signal to the clock input end of each of the second triggers in the register, so that the N-bit gate selection control signal in the register is output in parallel to each control end in the gate selection switch, wherein the gate selection control signal includes a first gate selection control signal and / or a second gate selection control signal; The shift register is used to shift and store the N-bit serial gate selection control signal input by the first FPGA unit into the register under the drive of the N first clock pulse signals; The register is used to output the N-bit gate selection control signal after shift registration in parallel to the control ends of the N gate selection switches under the drive of the second clock pulse signal.

[0010] Optionally, the second FPGA module includes a second FPGA unit, a first DAC unit, a second DAC unit and an ADC unit, wherein: The first end of the second FPGA unit is electrically connected to the host computer; The second end of the second FPGA module is electrically connected to the first end of the first DAC unit, the first end of the second DAC unit, and the first end of the ADC unit respectively; The second end of the first DAC unit is electrically connected to the input end of the gate selection switch; The second end of the second DAC unit is electrically connected to the input end of the row selection switch; The second end of the ADC unit is electrically connected to the output end of the first column selection switch; The third end of the second FPGA unit is electrically connected to the third end of the first FPGA unit; The first DAC unit is used to convert the digital signal output by the second FPGA unit into an analog signal, and send it to the gate signal terminal of the memristor array through the gate selection switch; The second DAC unit is used to convert the digital signal output by the second FPGA unit into an analog signal, and send it to the row signal terminal of the memristor array through the row selection switch; The ADC unit is used to convert the analog signal output by the column signal end of the memristor array through the first column selection switch into a digital signal, and send it to the second FPGA unit.

[0011] Optionally, the second FPGA module further includes a transimpedance amplifier, the transimpedance amplifier includes an amplifier, a feedback resistor component, a first switch component and a second switch component, and the feedback resistor component includes a plurality of feedback resistors with different resistance values; The negative input terminal of the transimpedance amplifier is electrically connected to the output terminal of the first column selection switch; The positive input terminal of the transimpedance amplifier is grounded; The output end of the transimpedance amplifier is electrically connected to the first end of each first switch in the first switch component; A first end of each feedback resistor in the feedback resistor assembly is electrically connected to a negative input terminal of the transimpedance amplifier; The second end of each feedback resistor in the feedback resistor assembly is electrically connected to the second end of each first switch in the switch assembly and the first end of each second switch in the second switch assembly respectively; A second end of each second switch in the second switch assembly is electrically connected to a second end of the ADC unit; The second FPGA unit is further used to send a resistance control signal to the first switch component and the second switch component according to a preset amplification factor, so that a first switch in the first switch component and a second switch in the second switch component are closed, wherein the first switch and the second switch are switches electrically connected to the resistor corresponding to the preset amplification factor; The first switch component is used to control one of the first switches in the first switch component to be closed and the other first switches to be opened after receiving the resistance control signal; The second switch component is used to control one of the second switches in the second switch component to be closed and the other second switches to be opened after receiving the resistance control signal.

[0012] The present application also provides an image recognition method, which is applied to an image recognition device, wherein the image recognition device includes a control processing module, a gate selection switch, a row selection switch, a first column selection switch, and a memristor array, wherein the memristor array includes a plurality of first memristors arranged in an array, and the method includes: The control processing module obtains the resistance values ​​of multiple second memristors with different resistance values ​​and the resistance values ​​of the second column selection switches, inputs a test voltage to the row signal terminal of each of the second memristors, receives a test current corresponding to the resistance value of each of the second memristors through the second column selection switch, and obtains a current-resistance relationship according to the resistance values ​​of the multiple second memristors, the resistance values ​​of the second column selection switches and the multiple test currents, wherein the multiple second memristors are memristors with known parameters, the second column selection switch is a column selection switch with known parameters, the control processing module is electrically connected to the row signal terminal of the second memristor and the output terminal of the second column selection switch, respectively, and the column signal terminal of the second memristor is electrically connected to the input terminal of the second column selection switch; The control processing module converts the pixel values ​​of the image to be identified to obtain the voltage to be identified, determines the row and column information of the memristor subarray to be operated in the memristor array according to the image to be identified, and obtains the weight voltage corresponding to the conductance value of each first memristor in the memristor subarray according to the conductance value of each first memristor and the current-resistance relationship, wherein the conductance value of each first memristor is mapped to the weight matrix of the neural network determined according to the image to be identified; The control processing module controls the closure of the gate selection switch, the row selection switch and the switch in the first column selection switch based on the row and column information of the memristor subarray, inputs the first gate voltage to the gate signal terminal of the memristor subarray, inputs each of the weight voltages to the row signal terminal of the memristor subarray accordingly, so that the weights in the weight matrix of the neural network are written into the memristor subarray, inputs the voltage to be identified into each row signal terminal of the memristor subarray, receives each column current output from each column signal terminal of the memristor subarray through each first column selection subswitches in the first column selection switch, and determines the recognition result of the image to be identified based on each of the column currents.

[0013] Optionally, the method further comprises: The control processing module reads the to-be-modulated row and column information in the memristor array in the weight modulation instruction, each target resistance value and each pulse voltage corresponding to the to-be-modulated row and column information, selects the currently modulated first memristor from the memristor array based on the to-be-modulated row and column information, and controls the gate selection switch, the row selection switch and the first column selection switch corresponding to the currently modulated first memristor to be closed; The control processing module inputs the second gate voltage to the gate signal end of the currently modulated first memristor, inputs the pulse voltage corresponding to the row and column information of the currently modulated first memristor to the row signal end of the currently modulated first memristor, receives the modulated current output by the column signal end of the currently modulated first memristor and then flows through the first column selection sub-switch, obtains the modulation resistance value according to the relationship between the modulation current and the current-resistance formula, determines whether the modulation resistance value is equal to the target resistance value, and if not, adjusts the pulse voltage so that the adjusted modulation resistance value is equal to the target resistance value, until the modulation resistance value of each of the first memristors corresponding to the row and column information to be modulated is equal to the corresponding target resistance value.

[0014] Optionally, the control processing module includes a host computer, a first FPGA unit and a second FPGA module; The control processing module converts the pixel value of the image to be recognized to obtain the voltage to be recognized, determines the row and column information of the memristor subarray to be operated in the memristor array according to the image to be recognized, and obtains the weight voltage corresponding to the conductance value of each first memristor in the memristor subarray according to the conductance value of each first memristor and the current-resistance relationship, wherein the conductance value of each first memristor is mapped to the weight matrix of the neural network determined according to the image to be recognized, including: The host computer converts the pixel values ​​of the image to be identified to obtain the voltage to be identified, determines the row and column information of the memristor subarray to be operated in the memristor array according to the image to be identified, obtains the weight voltage corresponding to the conductance value of each first memristor according to the conductance value of each first memristor in the memristor subarray and the current-resistance relationship, generates an image recognition instruction according to the voltage to be identified, the row and column information of the memristor subarray and the weight voltage of each first memristor, and sends it to the first FPGA unit and the second FPGA module, wherein the conductance value of each first memristor is mapped to the weight matrix of the neural network determined according to the image to be identified; The control processing module controls the gate selection switch, the row selection switch and the switch in the first column selection switch to close based on the row and column information of the memristor subarray, inputs the first gate voltage to the gate signal terminal of the memristor subarray, and inputs each of the weight voltages to the row signal terminal of the memristor subarray accordingly, so that the weights in the weight matrix of the neural network are written into the memristor subarray, inputs the voltage to be identified to each row signal terminal of the memristor subarray, receives each column current outputted by each column signal terminal of the memristor subarray through each first column selection subswitches in the first column selection switch, and determines the recognition result of the image to be identified according to each of the column currents, including: The first FPGA unit receives the image recognition instruction, reads the row and column information of the memristor subarray in the image recognition instruction, selects the row and column information of the currently operated first memristor from the row and column information of the memristor subarray, sends the row and column information of the currently operated first memristor to the second FPGA module, sends a first gate selection control signal to the gate selection switch, sends a first row selection control signal to the row selection switch, and sends a first column selection control signal to the first column selection switch; The gate selection switch closes a gate selection sub-switch in the gate selection switch according to the first gate selection control signal; The row selection switch closes a row selection sub-switch in the row selection switch according to the first row selection control signal; The first column selection switch closes a first column selection sub-switch in the first column selection switch according to the first column selection control signal; The second FPGA module receives the image recognition instruction, reads the voltage to be recognized, the row and column information of the memristor subarray, and the weight voltage of each of the first memristors in the image recognition instruction, inputs the first gate voltage to the gate signal terminal of the currently operating first memristor, and inputs the weight voltage corresponding to the row and column information of the currently operating first memristor to the row signal terminal of the currently operating first memristor; After the row signal end of each first memristor of the memristor subarray writes the corresponding weight voltage, the first FPGA unit sends a second gate selection control signal to the gate selection switch based on the row and column information of the memristor subarray, sends a second row selection control signal to the row selection switch, and sends a second column selection control signal to the first column selection switch; The gate selection switch closes a plurality of gate selection sub-switches in the gate selection switch according to the second gate selection control signal; The row selection switch closes a plurality of row selection sub-switches in the row selection sub-switches according to the second row selection control signal; The first column selection switch closes a plurality of column selection switches in the first column selection switch according to the second column selection control signal; The second FPGA module inputs the first gate voltage into the gate signal terminal of the memristor subarray, inputs the voltage to be identified into each row signal terminal of the memristor subarray, receives each column current output from each column signal terminal of the memristor subarray through each first column selection sub-switches in the first column selection switch, determines the recognition result of the image to be identified based on each column current, and sends the recognition result to the host computer.

[0015] Compared with the prior art, the present application provides an image recognition device and method, which obtains a current-resistance relationship according to the resistance values ​​of multiple second memristors, the resistance value of a second column selection switch and multiple test currents, obtains a weight voltage corresponding to the conductance value of each first memristor according to the conductance value of each first memristor in the memristor subarray and the current-resistance relationship, obtains a voltage to be recognized by converting the pixel value of the image to be recognized, controls the closure of the gate selection switch, the row selection switch and the first column selection switch based on the row and column information of the memristor subarray, inputs the first gate voltage to the gate signal terminal of the memristor subarray, and inputs each weight voltage to the row signal terminal of the memristor subarray accordingly, so that the weights in the weight matrix of the neural network are written into the memristor subarray, inputs the voltage to be recognized into each row signal terminal of the memristor subarray, and transmits the voltage to the first column selection switch through each first column selection switch in the first column selection switch. The switch receives each column current output by each column signal end of the memristor subarray, and determines the recognition result of the image to be recognized based on each column current. In the present application, since the current-resistance relationship is obtained based on the resistance values ​​of the multiple second memristors, the resistance value of the second column selection switch and the multiple test currents, the resistance value when the column selection switch is turned on is taken into account, so that after the weight voltage obtained based on the conductance value of the first memristor and the current-resistance relationship is input to the row signal end of the memristor subarray, the conductance value of the first memristor and the weight of the neural network can be accurately mapped. Moreover, since the multiple column currents output from the multiple column signal ends of the memristor subarray are the multiplication and accumulation of the voltage to be recognized and the conductance value performed by the memristor subarray on the column, the accuracy of the image recognition result determined based on the multiple column currents is very high, thereby making the accuracy of image recognition performed by the image recognition device and method of the present application very high. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A structural block diagram of an image recognition device disclosed in an embodiment of the present application; Figure 2 A circuit diagram of a memristor array disclosed in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a first memristor disclosed in an embodiment of the present application; Figure 4 A schematic diagram of the structure of an image recognition device disclosed in an embodiment of the present application; Figure 5A schematic diagram of the connection relationship between the control processing module disclosed in the embodiment of the present application, the second memristor, and the second column selection switch; Figure 6 The curve obtained by fitting the sum of the resistance R of each second memristor and the resistance R1 of the second column selection switch (ie, R+R1) with the test current value i disclosed in the embodiment of the present application; Figure 7 A schematic diagram of an image recognition device disclosed in an embodiment of the present application recognizing a handwritten digital image; Figure 8 A structural block diagram of another image recognition device disclosed in an embodiment of the present application; Fig. 9 A circuit diagram of a first serial-to-parallel conversion module disclosed in an embodiment of the present application; Fig.10 The data waveform diagram of each port of the first serial-to-parallel conversion module disclosed in the embodiment of the present application; Fig.11 A circuit diagram of a transimpedance amplifier disclosed in an embodiment of the present application; Fig.12 This is a flowchart of an image recognition method disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] It should be noted that when an element is referred to as being "fixed on" or "set on" another element, it can be directly on the other element or indirectly set on the other element; when an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

[0020] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0021] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "multiple" and "several" mean two or more, unless otherwise clearly and specifically defined.

[0022] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the conditions under which this application can be implemented. Therefore, they have no substantive technical significance. Any structural modification, change in proportional relationship or adjustment of size should still fall within the scope of the technical content disclosed in this application without affecting the effects and purposes that can be achieved by this application.

[0023] like Figures 1 to 5As shown, an embodiment of the present application provides an image recognition device, which includes a control processing module, a gate selection switch, a row selection switch, a first column selection switch and a memristor array, wherein the memristor array includes a plurality of first memristors arranged in an array; the control processing module is electrically connected to the input end of the gate selection switch, the input end of the row selection switch and the output end of the first column selection switch respectively; the output end of the gate selection switch is electrically connected to the gate signal end of the memristor array; the output end of the row selection switch is electrically connected to the row signal end of the memristor array; the input end of the first column selection switch is electrically connected to the column signal end of the memristor array; the control processing module A block is used to obtain the resistance values ​​of multiple second memristors with different resistance values ​​and the resistance value of the second column selection switch, input a test voltage to the row signal end of each second memristor, receive a test current corresponding to the resistance value of each second memristor through the second column selection switch, and obtain a current-resistance relationship according to the resistance values ​​of the multiple second memristors, the resistance value of the second column selection switch, and the multiple test currents, wherein the multiple second memristors are memristors with known parameters, the second column selection switch is a column selection switch with known parameters, and the control processing module is respectively connected to the row signal end of the second memristor and the output end and the control processing module of the second column selection switch. The control terminal is electrically connected, and the column signal terminal of the second memristor is electrically connected to the input terminal of the second column selection switch; the control processing module is also used to convert the pixel value of the image to be identified to obtain the voltage to be identified, determine the row and column information of the memristor subarray to be operated in the memristor array according to the image to be identified, and obtain the weight voltage corresponding to the conductance value of each first memristor according to the conductance value and current resistance relationship of each first memristor in the memristor subarray, wherein the conductance value of each first memristor is mapped to the weight matrix of the neural network determined according to the image to be identified; the control processing module is also used based on The row and column information of the memristor subarray controls the closure of the gate selection switch, the row selection switch and the sub-switches in the first column selection switch, inputs the first gate voltage to the gate signal terminal of the memristor subarray, and inputs each weight voltage to the row signal terminal of the memristor subarray accordingly, so that the weights in the weight matrix of the neural network are written into the memristor subarray, inputs the voltage to be identified into each row signal terminal of the memristor subarray, receives each column current outputted from each column signal terminal of the memristor subarray through each first column selection sub-switches in the first column selection switch, and determines the recognition result of the image to be identified according to each column current.

[0024] In this embodiment, Figure 2As shown, the memristor array includes a plurality of first memristors arranged in an array, and the first memristor may be a three-terminal memristor having three ports, namely, a gate signal terminal (or control terminal) Vg, an input terminal Vin (or a row signal terminal BL), and an output terminal Iout (or a column signal terminal SL), wherein the control signal adjusts and changes the state (such as the open and closed state) of the first memristor through the gate signal terminal Vg, and the working state of the first memristor in the nonlinear region can be adjusted and manipulated by changing the size and frequency of the control signal. The input signal is generally input into the first memristor through the input terminal Vin, and a voltage or current signal is usually applied as an input, and the output terminal Iout outputs corresponding current information according to the input signal and the characteristics of the memristor. Compared with two-terminal memristors, three-terminal memristors have independent read / write circuits and selection circuits, which means that each memristor can be accessed individually without being affected by other memristors. This independence makes the memristor array composed of three-terminal memristors more flexible and allows parallel storage operations on multiple three-terminal memristors at the same time. At the same time, the control terminal of the three-terminal memristor can control and adjust the resistance value of the three-terminal memristor to adapt to different scene requirements. Since each three-terminal memristor has its own read / write circuit and selection circuit, the current or voltage interference between adjacent three-terminal memristors is reduced a lot, which improves the system's anti-interference performance and reduces the risk of misoperation. Figure 2 As shown, the scale of the memristor array is n×n, where n is a positive integer greater than 1. The memristor array has three ports, the first port is the gate signal terminal Vg1-Vgn, the second port is the row signal terminal BL1-BLn, and the third port is the column signal terminal SL1-SLn. The input terminal of the three-terminal memristor in each row is horizontally connected to BL1-BLn, the control terminal of the three-terminal memristor in each column is vertically connected to Vg1-Vgn, and the output terminal of the three-terminal memristor in each column is vertically connected to SL1-SLn. The three-terminal memristor array uses a three-terminal structure to better isolate the interference between each memristor, reduce the error caused by factors such as interconnected circuits and temperature changes, and at the same time, the working state of the memristor can be adjusted through the control terminal, so that the three-terminal memristor is more flexibly applicable to different application requirements. The three-terminal memristor array composed of multiple three-terminal memristors only needs to activate the required memristors during use, without activating the entire array, which can greatly reduce power consumption and improve energy efficiency.

[0025] In an existing image recognition device based on a memristor array, it is generally necessary to select a memristor in the memristor array that needs to be written with a weight of a neural network through a gate selection switch, a row selection switch, and a column selection switch, so that the conductance value mapped with the weight of the neural network is written into the corresponding memristor. Generally, a voltage is applied to the row signal end of the current memristor in the memristor array, and according to the current value output by the column signal end of the current memristor of the memristor array after passing through the column selection switch and the applied voltage value, it is determined whether the conductance value of the current memristor is equal to the conductance value to be written. However, since the column selection switch connected to the column signal end of the memristor generally adopts a multi-way selection switch, such as a four-to-one selection switch, the multi-way selection switch has a resistance when it is turned on, and the resistance of the multi-way selection switch when it is turned on is about 100-1000Ω. Since the resistance of the column selection switch connected to the column signal end of the memristor is generally not considered when mapping the conductance value of the memristor with the weight value of the neural network in the prior art, it is impossible to achieve accurate mapping of the conductance value of the memristor with the weight value of the neural network. To solve this problem, in this embodiment, Figure 5 As shown, the second memristor is a three-terminal memristor with a known resistance value, the second column selection switch is a column selection switch with a known resistance value, a plurality of second memristors with different resistance values ​​and known resistance values ​​are prepared, the control processing module first obtains the resistance values ​​of the plurality of second memristors with different resistance values ​​and the resistance value of the second column selection switch stored in advance, first selects the second memristor currently under test from the plurality of second memristors and connects it between the second column selection switch and the control processing module, the control processing module controls one of the second column selection sub-switches connected to the second memristor currently under test in the second column selection switch to be closed, and then inputs the test voltage to the row signal end of the second memristor currently under test, and the control processing module receives the test voltage corresponding to the resistance value of the second memristor currently under test through the second column selection switch. The current is converted into a test current value for storage, and in this way, the test current value corresponding to the second memristor with each resistance value is obtained. For example, 9 second memristors with resistance values ​​of 1200Ω, 2000Ω, 3000Ω, 4300Ω, 5100Ω, 6200Ω, 7500Ω, 9100Ω, and 10000Ω can be used, and the test current values ​​received from the second column selection switch are 111uA, 77uA, 55uA, 41uA, 35uA, 29uA, 25uA, 20uA, and 18uA respectively. Then, the resistance value of each second memristor is added to the resistance value R1 (such as 600Ω) of the second column selection switch to obtain the sum of the resistance value of each second memristor and the resistance value of the second column selection switch, such as Figure 6 As shown, the least square method can be used to fit the sum of the resistance R of each second memristor and the resistance R1 of the second column selection switch (ie, R+R1) with the test current value i to obtain a fitted curve, and then the current-resistance relationship is obtained according to the curve, that is, the relationship between the resistance of the second memristor and the test current value. The specific current-resistance relationship is shown as follows: , Wherein, R is the resistance value of the second memristor, R1 is the resistance value of the second column selection switch, and i is the test current value.

[0026] In this embodiment, the control processing module further converts the pixel values ​​of the image to be identified to obtain the converted voltage to be identified. For example, if the image to be identified is a grayscale image with pixel values ​​in the range of 0 to 255, the pixel values ​​of 0 to 255 can be encoded as a pulse string of 8 bits and an amplitude of 0.2V, which is the converted voltage to be identified, or the pixel values ​​of 0 to 255 are mapped to 0 to 0.2V to obtain the converted voltage to be identified; the row and column information of the memristor subarray to be operated in the memristor array is determined according to the image to be identified, the weight matrix of the neural network is determined according to the image to be identified, the conductance value of each first memristor in the memristor subarray is determined according to the weight matrix of the neural network, and then the target resistance value of each first memristor is obtained according to the conductance value of each first memristor, which can be obtained from the memristor array. The current first memristor is selected in the resistor array, and the sub-switch corresponding to the current first memristor in the gate selection switch, the row selection switch and the first column selection switch is controlled to be closed, and the gate voltage is input to the gate signal end of the current first memristor, and the pulse voltage is input to the row signal end of the current first memristor, and the current output by the column signal end of the current first memristor is received, and the current value is obtained according to the received current, and the resistance value is obtained according to the relationship between the current value and the current resistance, and it is judged whether the resistance value is equal to the target resistance value of the first memristor. If not, the pulse voltage is adjusted so that the adjusted resistance value is equal to the target resistance value of the first memristor, and the adjusted pulse voltage is determined to be the weight voltage corresponding to the conductance value of the first memristor, and the weight voltage corresponding to the conductance value of each first memristor is obtained in this way.

[0027] In this embodiment, Figure 4As shown, a plurality of gate selection sub-switches are provided in the gate selection switch, a plurality of row selection sub-switches are provided in the row selection switch, a plurality of first column selection sub-switches are provided in the first column selection switch, the gate selection switch, the row selection switch and the first column selection switch are all multi-way selection switches, such as a four-to-one selection switch, the control processing module can select the row and column information of the currently operated first memristor from the row and column information of the memristor sub-array, and send the first gate selection control signal, the first row selection control signal and the first column selection control signal to the gate selection switch, the row selection switch and the first column selection switch respectively according to the row and column information of the currently operated first memristor, so that the gate selection switch closes the gate selection sub-switch connected to the gate signal end of the currently operated first memristor, the row selection switch closes the row selection sub-switch connected to the row signal end of the currently operated first memristor, and the first column selection switch closes the first column selection sub-switch connected to the column signal end of the currently operated first memristor, and then inputs the first gate voltage to the gate signal end of the currently operated first memristor through the closed gate selection sub-switch, so that the currently operated The first memristor to be operated is turned on, and the weight voltage is input into the row signal end of the first memristor to be operated through the closed row selection sub-switch, so that the conductance value of the currently operated first memristor is written, until the conductance value corresponding to each first memristor in the memristor sub-array is written, that is, the weight in the weight matrix of the neural network is written into the memristor sub-array, and then the first gate voltage is input into the gate signal end of the memristor sub-array, so that each first memristor in the memristor sub-array is turned on, and the voltage to be identified obtained by converting the pixel value of the image to be identified is input into each row signal end of the memristor sub-array, and each column current outputted by each column signal end of the memristor sub-array is the result of multiplying the voltage to be identified with the conductance value of the first memristor on each column, that is, the result of multiplying the pixel value of the image to be identified with the single column weight in the weight matrix of the neural network, and each column current value is obtained according to each column current after flowing through each first column selection sub-switches, and the recognition result of the image to be identified can be determined by comparing the magnitude of each column current value. In the present application, since the resistance values ​​of the first column selection sub-switch and the second column selection switch are the same, the current-resistance relationship obtained according to the resistance values ​​of the multiple second memristors, the resistance values ​​of the second column selection switches and the multiple test currents takes into account the resistance value of the second column selection switch when it is turned on, so that the weight voltage obtained according to the conductance value of the first memristor and the current-resistance relationship takes into account the resistance value of the first column selection sub-switch when it is turned on. After the weight voltage is input to the row signal end of the memristor sub-array, the conductance value of the first memristor and the weight of the neural network can be accurately mapped. Moreover, since the multiple column currents output from the multiple column signal ends of the memristor sub-array are the multiplication and accumulation of the voltage to be identified and the conductance value performed by the memristor sub-array on the column, the accuracy of the image recognition result determined according to the multiple column currents is very high, thereby making the accuracy of image recognition performed by the image recognition device and method of the present application very high.

[0028] In this embodiment, the principle of the image recognition device recognizing an image can be specifically as follows: Figure 1 and Figure 7As shown, for example, the image to be identified is a handwritten digital image, and the control processing module converts the pixel value of the image to be identified to obtain 9 pulse voltages Vin1-Vin9 to be identified. Since there are 10 possibilities for a single digit from 0 to 9, the rows and columns of the memristor subarray to be operated are 9 rows and 10 columns. For example, the row and column information of the memristor subarray to be operated is 01-90, the row and column information of the 10 first memristors in the first row is 01-10, and the row and column information of the 9 first memristors in the first column is 01, 11, 21, 31, 41, 51, 61, 71, 81. According to the 10 possibilities from 0 to 9, a group of weights corresponding to each digit from 0 to 9 in the neural network is determined, and the conductance value of each first memristor in the memristor subarray is determined according to the weight. For example, the conductance value of the first memristor in the first column (column information is 01, 11, 21, 31, 41, 51, 61, 71, 81) is determined according to a group of weights corresponding to 0. Then, according to the conductance of each first memristor, the target resistance value of each first memristor is obtained. First, the row and column information of 01 is selected from the memristor array as the current first memristor), and the sub-switch corresponding to the current first memristor in the gate selection switch, the row selection switch and the first column selection switch is controlled to be closed, and the gate voltage is input to the gate signal end of the current first memristor, and the pulse voltage is input to the row signal end of the current first memristor. The current output by the column signal end of the current first memristor is received, and the current value is obtained according to the received current. According to the relationship between the current value and the current resistance, the resistance value is obtained, and it is judged whether the resistance value is equal to the target resistance value of the first memristor. If not, the pulse voltage is adjusted so that the adjusted resistance value is equal to the target resistance value of the first memristor, and the adjusted pulse voltage is determined to be the weight voltage corresponding to the conductance value of the first memristor. In this way, a group of weight voltages of the first memristor in the first column are obtained, until ten groups of weight voltages from the first column to the tenth column are obtained;Then select the current first memristor from the memristor array (for example, first select the row and column information as 01 as the current first memristor), control the gate selection switch, the row selection switch and the sub-switch corresponding to the current first memristor in the first column selection switch to close, input the gate voltage to the gate signal end of the current first memristor, input the weight voltage corresponding to the current first memristor (for example, the weight voltage corresponding to the first memristor with the row and column information as 01) to the row signal end of the current first memristor, and complete writing the conductance value corresponding to the row and column information of 01 in the memristor subarray into the first memristor, in this way, map a group of weights corresponding to 0 to the first column (column information is 01, 11, 21, 31, 4 The conductance value of the first memristor (1, 51, 61, 71, 81) is written into the first memristor in the first column, and then a group of weights corresponding to 1 is mapped to the conductance value of the first memristor in the second column (column information is 02, 12, 22, 32, 42, 52, 62, 72, 82) and written into the first memristor in the first column. In this way, a group of weights corresponding to 6 is mapped to the seventh column (column information is 07, 17, 27, 37, 47, 57, 67, 77, 87), until the weights corresponding to 0-9 are all written into the corresponding first memristor, and then 9 pulse voltages Vin1-Vin9 are respectively input into BL1-BL9 of the memristor subarray (Vin 1 inputs BL1... Vin9 inputs BL9), the SL1 column current output from the first column signal terminal SL1 of the memristor subarray is the result of multiplying the voltage to be identified Vin1-Vin9 with the conductance values ​​of the first nine memristors in the first column (column information is 01, 11, 21, 31, 41, 51, 61, 71, 81), that is, the result of multiplying the pixel value of the image to be identified with the weight corresponding to 0, and the SL7 column current output from the seventh column signal terminal SL7 of the memristor subarray is the result of multiplying the voltage to be identified Vin1-Vin9 with the conductance values ​​of the first nine memristors in the seventh column (column information is 07, 17, 27, 37, 47, 57, 67 , 77, 87) of the 9 first memristors, that is, the pixel value of the image to be identified is multiplied and added with the weight corresponding to 6. The control processing module receives the first column current to the tenth column current after the SL1 column current to the SL10 column current output from the first column signal terminal SL1-SL10 of the memristor subarray flows through each first column selection sub-switch, converts the first column current to the tenth column current into the first column current value to the tenth column current value, compares the first column current value to the tenth column current value, and according to the comparison result, it can be known that the seventh column current value is greater than the other column current values, and the control processing module determines that the number in the image to be identified is 6. In this embodiment, the control processing module, the gate selection switch, the row selection switch and the first column selection switch are not attached; Figure 7 is displayed in the

[0029] Compared with the prior art, the present application provides an image recognition device and method, which obtains a current-resistance relationship according to the resistance values ​​of multiple second memristors, the resistance value of a second column selection switch and multiple test currents, obtains a weight voltage corresponding to the conductance value of each first memristor according to the conductance value of each first memristor in the memristor subarray and the current-resistance relationship, obtains a voltage to be recognized by converting the pixel value of the image to be recognized, controls the gate selection switch, the row selection switch and the sub-switches in the first column selection switch to be closed based on the row and column information of the memristor subarray, inputs the first gate voltage to the gate signal terminal of the memristor subarray, and inputs each weight voltage to the row signal terminal of the memristor subarray accordingly, so that the weights in the weight matrix of the neural network are written into the memristor subarray, inputs the voltage to be recognized into each row signal terminal of the memristor subarray, and controls the pixel value of the image to be recognized by converting the pixel value of the image to be recognized. The switch receives each column current output by each column signal end of the memristor subarray, and determines the recognition result of the image to be recognized based on each column current. In the present application, since the current-resistance relationship is obtained based on the resistance values ​​of the multiple second memristors, the resistance value of the second column selection switch and the multiple test currents, the resistance value when the column selection switch is turned on is taken into account, so that after the weight voltage obtained based on the conductance value of the first memristor and the current-resistance relationship is input to the row signal end of the memristor subarray, the conductance value of the first memristor and the weight of the neural network can be accurately mapped. Moreover, since the multiple column currents output from the multiple column signal ends of the memristor subarray are the multiplication and accumulation of the voltage to be recognized and the conductance value performed by the memristor subarray on the column, the accuracy of the image recognition result determined based on the multiple column currents is very high, thereby making the accuracy of image recognition performed by the image recognition device and method of the present application very high.

[0030] As an implementation mode, in an embodiment of the present application, the control processing module is also used to read the row and column information to be modulated in the memristor array in the weight modulation instruction, each target resistance corresponding to the row and column information to be modulated, and each pulse voltage, and select the currently modulated first memristor from the memristor array based on the row and column information to be modulated, and control the gate selection switch, the row selection switch and the first column selection switch. The sub-switch corresponding to the currently modulated first memristor is closed; the control processing module is also used to input the second gate voltage to the gate signal end of the currently modulated first memristor, input the pulse voltage corresponding to the row and column information of the currently modulated first memristor to the row signal end of the currently modulated first memristor, receive the modulation current after the current output from the column signal end of the currently modulated first memristor flows through the first column selection sub-switch, obtain the modulation resistance according to the relationship between the modulation current and the current resistance, and judge whether the modulation resistance is equal to the target resistance. If not, adjust the pulse voltage so that the adjusted modulation resistance is equal to the target resistance, until the modulation resistance of each first memristor corresponding to the row and column information to be modulated is equal to the corresponding target resistance.

[0031] In this embodiment, the control processing module reads the row and column information to be modulated in the memristor array in the weight modulation instruction, each target resistance value and each pulse voltage corresponding to the row and column information to be modulated, selects the first memristor currently modulated from the memristor array based on the row and column information to be modulated, controls the gate selection switch, the row selection switch and the sub-switch corresponding to the first memristor currently modulated in the first column selection switch to be closed, inputs the second gate voltage to the gate signal end of the first memristor currently modulated, so that the first memristor currently modulated is turned on, inputs the pulse voltage corresponding to the row and column information of the first memristor currently modulated, inputs the pulse voltage corresponding to the row and column information of the first memristor currently modulated, inputs the current output from the column signal end of the first memristor currently modulated, receives the current and flows through the first column selection sub-switch, and then controls the gate selection switch, the row selection switch and the sub-switch corresponding to the first memristor currently modulated. The modulation current after the switch is obtained by obtaining the modulation resistance according to the relationship between the modulation current and the current resistance, and judging whether the modulation resistance is equal to the target resistance. If the modulation resistance is not equal to the target resistance, the pulse voltage is adjusted so that the adjusted modulation resistance obtained according to the relationship between the adjusted modulation current outputted from the column signal end of the currently modulated first memristor and the current resistance is equal to the target resistance. Then, the resistance of other first memristors in the row and column information to be modulated is modulated in this way until the modulation resistance of each first memristor corresponding to the row and column information to be modulated is equal to the corresponding target resistance. The weight modulation operation is completed by executing the weight modulation instruction, which can realize the automatic operation of the weight mapping of the neural network and greatly improve the efficiency of weight deployment.

[0032] like Figure 8As shown, as an implementation mode, in the embodiment of the present application, the control processing module includes a host computer, a first FPGA unit and a second FPGA module; the host computer is electrically connected to the first end of the first FPGA unit and the first end of the second FPGA module respectively; the second end of the first FPGA unit is electrically connected to the second end of the second FPGA module; the third end of the first FPGA unit is electrically connected to the control end of the gate selection switch; the fourth end of the first FPGA unit is electrically connected to the control end of the row selection switch; the fifth end of the first FPGA unit is electrically connected to the control end of the first column selection switch; the third end of the second FPGA module is electrically connected to the input end of the gate selection switch; the fourth end of the second FPGA module is electrically connected to the control end of the row selection switch The input end is electrically connected; the fifth end of the second FPGA module is electrically connected to the output end of the first column selection switch; the output end of the gate selection switch is electrically connected to the gate signal end of the memristor array; the output end of the row selection switch is electrically connected to the row signal end of the memristor array; the input end of the first column selection switch is electrically connected to the column signal end of the memristor array; the host computer is used to convert the pixel value of the image to be identified, obtain the voltage to be identified, determine the row and column information of the memristor subarray to be operated in the memristor array according to the image to be identified, and obtain the weight voltage corresponding to the conductance value of each first memristor in the memristor subarray according to the conductance value and current-resistance relationship of each first memristor, and determine the weight voltage corresponding to the conductance value of each first memristor according to the voltage to be identified, the row and column information of the memristor subarray and The weight voltage of each first memristor generates an image recognition instruction and sends it to the first FPGA unit and the second FPGA module, wherein the conductance value of each first memristor is mapped to the weight matrix of the neural network determined according to the image to be recognized; the first FPGA unit is used to receive the image recognition instruction, read the row and column information of the memristor subarray in the image recognition instruction, select the row and column information of the currently operated first memristor from the row and column information of the memristor subarray, send the row and column information of the currently operated first memristor to the second FPGA module, send the first gate selection control signal to the gate selection switch, send the first row selection control signal to the row selection switch, and send the first column selection control signal to the first column selection switch; the gate selection A switch, used to close a gate selection sub-switch in the gate selection switch according to a first gate selection control signal; a row selection switch, used to close a row selection sub-switch in the row selection switch according to a first row selection control signal; a first column selection switch, used to close a first column selection sub-switch in the first column selection switch according to a first column selection control signal; a second FPGA module, used to receive an image recognition instruction, read the voltage to be recognized, the row and column information of the memristor subarray and the weight voltage of each first memristor in the image recognition instruction, input the first gate voltage to the gate signal end of the currently operating first memristor, and input the weight voltage corresponding to the row and column information of the currently operating first memristor to the row signal end of the currently operating first memristor;The first FPGA unit is also used to send a second gate selection control signal to the gate selection switch, send a second row selection control signal to the row selection switch, and send a second column selection control signal to the first column selection switch based on the row and column information of the memristor subarray until the row signal terminal of each first memristor of the memristor subarray is written with the corresponding weight voltage; the gate selection switch is also used to close multiple gate selection sub-switches in the gate selection switch according to the second gate selection control signal; the row selection switch is also used to close multiple row selection sub-switches in the row selection sub-switch according to the second row selection control signal; the first column selection switch is also used to close multiple first column selection sub-switches in the first column selection switch according to the second column selection control signal; the second FPGA module is also used to input the first gate voltage into the gate signal terminal of the memristor subarray, input the voltage to be identified into each row signal terminal of the memristor subarray, receive each column current outputted by each column signal terminal of the memristor subarray through each first column selection sub-switches in the first column selection switch, determine the recognition result of the image to be identified according to each column current, and send the recognition result to the host computer. ;

[0033] In this embodiment, the model of the FPGA (Field-Programmable Gate Array), the first FPGA unit is FPGA XC7S50.

[0034] In this embodiment, the host computer converts the pixel values ​​of the image to be identified to obtain the converted voltage to be identified, determines the row and column information of the memristor subarray to be operated in the memristor array according to the image to be identified, determines the weight matrix of the neural network according to the image to be identified, determines the conductance value of each first memristor in the memristor subarray according to the weight matrix of the neural network, and then obtains the target resistance value of each first memristor according to the conductance value of each first memristor, selects the current first memristor from the memristor array, controls the gate selection switch, the row selection switch and the sub-switch corresponding to the current first memristor in the first column selection switch to be closed, inputs the gate voltage to the gate signal end of the current first memristor, inputs the pulse voltage to the row signal end of the current first memristor, and receives The current output by the column signal end of the current first memristor is obtained, and the current value is obtained according to the received current, and the resistance value is obtained according to the current value and the current-resistance relationship, and it is judged whether the resistance value is equal to the target resistance value of the first memristor. If not, the pulse voltage is adjusted so that the adjusted resistance value is equal to the target resistance value of the first memristor, and the adjusted pulse voltage is determined to be the weighted voltage corresponding to the conductance value of the first memristor. In this way, the weighted voltage corresponding to the conductance value of each first memristor is obtained, and according to the voltage to be identified, the row and column information of the memristor subarray and the weighted voltage of each first memristor, an image recognition instruction is generated and sent to the first FPGA unit and the second FPGA module, and the first FPGA unit receives the image recognition instruction, reads The image recognition instruction is obtained by taking the row and column information of the memristor subarray, selecting the row and column information of the currently operated first memristor from the row and column information of the memristor subarray, sending the row and column information of the currently operated first memristor to the second FPGA module, sending the first gate selection control signal to the gate selection switch, sending the first row selection control signal to the row selection switch, and sending the first column selection control signal to the first column selection switch. The gate selection switch closes a gate selection sub-switch in the gate selection switch according to the first gate selection control signal, the row selection switch closes a row selection sub-switch in the row selection switch according to the first row selection control signal, and the first column selection switch closes a first column selection sub-switch in the first column selection switch according to the first column selection control signal. The second FPGA module receives the image recognition instruction. Instruction, read the voltage to be identified, the row and column information of the memristor subarray and the weight voltage of each first memristor in the image recognition instruction, input the first gate voltage to the gate signal end of the currently operated first memristor through a closed gate selection subswitch, so that the currently operated first memristor is turned on, and input the weight voltage corresponding to the row and column information of the currently operated first memristor through a closed row selection subswitch to the row signal end of the currently operated first memristor, so that the conductance value of the currently operated first memristor is written, until the conductance value corresponding to each first memristor in the memristor subarray is written, that is, the weights in the weight matrix of the neural network are written into the memristor subarray, and after the corresponding weight voltage is written into the row signal end of each first memristor in the memristor subarray,The first FPGA unit sends a second gate selection control signal to the gate selection switch based on the row and column information of the memristor subarray, sends a second row selection control signal to the row selection switch, and sends a second column selection control signal to the first column selection switch. The gate selection switch closes multiple gate selection sub-switches in the gate selection switch according to the second gate selection control signal, and the row selection switch closes multiple row selection sub-switches in the row selection sub-switches according to the second row selection control signal. The first column selection switch closes multiple column selection sub-switches in the first column selection switch according to the second column selection control signal. The second FPGA module inputs the first gate voltage into the gate signal end of the memristor subarray to turn on each first memristor in the memristor subarray, and inputs the voltage to be identified obtained by converting the pixel value of the image to be identified into each row signal end of the memristor subarray. The column currents outputted from the column signal ends of the memristor subarray are the cumulative results of multiplying the voltage to be identified with the conductance value of the first memristor on each column, that is, the cumulative results of multiplying the pixel value of the image to be identified with the single column weight in the weight matrix of the neural network. Each column current after the first column selection sub-switch obtains each column current value. By comparing the size of each column current value, the recognition result of the image to be recognized can be determined, and the recognition result is sent to the host computer. In this application, by setting the first FPGA unit and the second FPGA module, the gate selection switch, the row selection switch and the first column selection switch are controlled by the first FPGA unit, and the gate voltage and the weight voltage are generated according to the data in the instruction by the second FPGA module, so that the switch control and the data control are separated to different FPGAs, which can enhance the isolation of the device, reduce the mutual influence between different functional modules, reduce possible interference and conflicts, and thus improve the stability and reliability of the system. The independent first FPGA unit is responsible for the switch control, which can realize the fast, accurate and flexible control of the first memristor in the three-terminal memristor array. According to actual needs, the switch state of the first memristor can be dynamically adjusted to realize different functional combinations and configurations. Such a design can improve the flexibility and scalability of the device, so that it can adapt to the needs of different application scenarios. ,

[0035] As an implementation mode, in an embodiment of the present application, the host computer is also used to send a weight modulation instruction, which includes the row and column information to be modulated in the memristor array, each target resistance value and each pulse voltage corresponding to the row and column information to be modulated; the first FPGA unit is used to receive the weight modulation instruction, read the row and column information to be modulated in the weight modulation instruction, select the current modulated first memristor from the memristor array based on the row and column information to be modulated, and send the corresponding second gate selection control signal to the gate selection switch, and send the second row selection control signal to the row selection switch, and send the second column selection control signal to the first column selection switch; the gate selection switch is used to close a gate selection sub-switch in the gate selection switch according to the second gate selection control signal; the row selection switch is used to close a row selection sub-switch in the row selection switch according to the second row selection control signal; the first column selection switch is used to close a first column selection sub-switch in the first column selection switch according to the second column selection control signal; the second FPGA module is used to receive the weight modulation instruction, read the row and column information to be modulated in the weight modulation instruction, and the row and column information to be modulated The target resistance values ​​and pulse voltages corresponding to the information are inputted into the gate signal end of the first memristor currently modulating, and the pulse voltage corresponding to the row and column information of the first memristor currently modulating is inputted into the row signal end of the first memristor currently modulating, and the modulated current after the current outputted from the column signal end of the first memristor currently modulating flows through the first column selector switch is received, and the modulated current is sent to the host computer; the host computer is also used to obtain the modulation resistance value according to the relationship between the modulation current and the current resistance, and judge whether the modulation resistance value is equal to the target resistance value, if not, send the voltage control signal to the second FPGA module; the second FPGA module is used to receive the voltage control signal, and based on the voltage control signal, input the adjusted pulse voltage to the row signal end of the first memristor currently modulating, so that the modulation resistance value is equal to the target resistance value according to the adjusted modulation resistance value, until the modulation resistance value of each first memristor corresponding to the row and column information to be modulated is equal to the corresponding target resistance value, wherein the adjusted modulation resistance value is obtained according to the relationship between the adjusted modulation current and the current resistance outputted from the column signal end of the first memristor currently modulating.

[0036] In this embodiment, the host computer sends a weight modulation instruction to the first FPGA unit and the second FPGA module, the weight modulation instruction includes the row and column information to be modulated in the memristor array, each target resistance value and each pulse voltage corresponding to the row and column information to be modulated, the first FPGA unit receives the weight modulation instruction, reads the row and column information to be modulated in the weight modulation instruction, selects the current modulated first memristor from the memristor array based on the row and column information to be modulated, and sends the corresponding second gate selection control signal to the gate selection switch, sends the second row selection control signal to the row selection switch, and sends the second column selection control signal To the first column selection switch, the gate selection switch closes a gate selection sub-switch in the gate selection switch according to the second gate selection control signal, the row selection switch closes a row selection sub-switch in the row selection switch according to the second row selection control signal, the first column selection switch closes a first column selection sub-switch in the first column selection switch according to the second column selection control signal, the second FPGA module receives the weight modulation instruction, reads the row and column information to be modulated in the weight modulation instruction, each target resistance value and each pulse voltage corresponding to the row and column information to be modulated, and inputs the second gate voltage to the gate signal end of the currently modulated first memristor, so that the currently modulated The first memristor is turned on, and a pulse voltage corresponding to the row and column information of the current modulated first memristor is input into the row signal end of the current modulated first memristor, and the current output by the column signal end of the current modulated first memristor flows through the first column selector switch, and the modulated current is sent to the host computer. The host computer obtains the modulated resistance value according to the relationship between the modulated current and the current resistance, and determines whether the modulated resistance value is equal to the target resistance value. If the modulated resistance value is not equal to the target resistance value, a voltage control signal is sent to the second FPGA module. The second FPGA module receives the voltage control signal and, based on the voltage control signal, The adjusted pulse voltage is input to the row signal end of the currently modulated first memristor, so that the adjusted modulation resistance obtained according to the relationship between the adjusted modulation current and the current resistance outputted from the column signal end of the currently modulated first memristor is equal to the target resistance, and then the resistance of other first memristors in the row and column information to be modulated is modulated in this way until the modulation resistance of each first memristor corresponding to the row and column information to be modulated is equal to the corresponding target resistance, and the weight modulation operation is completed by executing the weight modulation instruction, which can realize the automatic operation of the weight mapping of the neural network and can greatly improve the efficiency of weight deployment.

[0037] like Figure 8 and Fig. 9As shown, as an implementation mode, in the embodiment of the present application, it also includes a first serial-to-parallel conversion module, the first serial-to-parallel conversion module includes a shift register and a register, the shift register includes N first triggers, the register includes N second triggers, and there are N gate select switches, where N is a positive integer greater than 5; the data input terminal of the shift register is electrically connected to the first sub-port in the third end of the first FPGA unit; the clock input terminal of each first trigger in the shift register is electrically connected to the second sub-port in the third end of the first FPGA unit; the reset input terminal of each first trigger in the shift register is electrically connected to the third sub-port in the third end of the first FPGA unit; the data output terminal of each first trigger in the shift register is electrically connected to the data input terminal of each second trigger of the register one by one; the clock input terminal of each second trigger in the register is electrically connected to the fourth sub-port in the third end of the first FPGA unit; the reset input terminal of each second trigger in the register is electrically connected to the fifth sub-port in the third end of the first FPGA unit; The data output end of the device is electrically connected to the control end of the N gate selection switches in a one-to-one correspondence; the first FPGA unit is used to convert the N-bit parallel gate selection control signal into an N-bit serial gate selection control signal and input it into the data input end of the shift register, and then input N first clock pulse signals to the clock input end of each first trigger in the shift register, so that the N-bit serial gate selection control signal is shifted and registered in the N second triggers in the register, and after the serial gate selection control signal is shifted and registered in the register, a second time pulse signal is sent to the clock input end of each second trigger in the register, so that the N-bit gate selection control signal in the register is output in parallel to each control end in the gate selection switch; the shift register is used to shift and register the N-bit serial gate selection control signal input by the first FPGA unit into the register under the drive of the N first clock pulse signals; the register is used to output the N-bit gate selection control signal after shifting and registering in parallel to the control end in the N gate selection switches under the drive of the second clock pulse signal, wherein the gate selection control signal includes the first gate selection control signal and / or the second gate selection control signal.

[0038] In this embodiment, the first FPGA unit converts the N-bit parallel gate selection control signal into an N-bit serial gate selection control signal and inputs it into the data input end of the shift register, and then inputs N first clock pulse signals to the clock input end of each first trigger in the shift register, so that the N-bit serial gate selection control signal is shifted and stored in the N second triggers in the register, and after the serial gate selection control signal is shifted and stored in the register, a second time pulse signal is sent to the clock input end of each second trigger in the register, so that the N-bit gate selection control signal in the register is output in parallel to each control end in the gate selection switch, wherein the gate selection control signal includes the first gate selection control signal and / or the second gate selection control signal. By setting the first serial-to-parallel conversion module, the first FPGA unit only needs to use 5 IO interfaces, and can output N gate selection control signals in parallel through the first serial-to-parallel conversion module, which can effectively reduce the IO interface pressure of the first FPGA unit and increase scalability.

[0039] like Fig. 9 As shown, in this embodiment, the first trigger and the second trigger can both be edge-triggered D triggers. Taking N as 8 as an example, the shift register includes 8 first triggers, the register includes 8 second triggers, and 8 gate select switches are provided. The data input terminal (SER terminal) of the first first trigger in the shift register is electrically connected to the first sub-port in the third end of the first FPGA unit, the clock input terminals (SRCLK terminals) of the 8 first triggers in the shift register are all electrically connected to the second sub-port in the third end of the first FPGA unit, the reset input terminals (SRCLR terminals) of the 8 first triggers in the shift register are electrically connected to the third sub-port in the third end of the first FPGA unit, and the data output terminals of each first trigger in the shift register are electrically connected to each of the registers. The data input terminals of the eight second flip-flops in the register are electrically connected one by one, the clock input terminals (RCLK terminals) of the eight second flip-flops in the register are all electrically connected to the fourth sub-port in the third terminal of the first FPGA unit, the reset input terminals (RCLR terminals) of the eight second flip-flops in the register are electrically connected to the fifth sub-port in the third terminal of the first FPGA unit, the data output terminals (QA-QH) of the eight second flip-flops in the register are electrically connected one by one to the control terminals in the eight gate select switches (i.e., gate select switch 1 to gate select switch 8), the output signal QH' of the eighth first flip-flop in the shift register can be used as a cascade signal. When a control signal needs to be added, it is only necessary to connect the QH' signal to the data input terminal of the first flip-flop of the shift register of the next serial-to-parallel conversion module, such as Fig.10As shown, when the 8-bit serial gate select control signal input to the SER end is specifically 10000100, the data output in parallel by the output ends QA, QB, QC, QD, QE, QF, QG, and QH of the 8 second triggers in the register are 0, 0, 1, 0, 0, 0, 0, and 1 respectively; the function of the first serial-to-parallel conversion module is to convert the received serial gate select control signal into a parallel gate select control signal output. By setting the first serial-to-parallel conversion module, the first FPGA unit only needs to use 5 IO interfaces, and 8 gate select control signals can be output in parallel through the first serial-to-parallel conversion module, which can effectively reduce the FPGA IO pressure and increase scalability.

[0040] like Figure 8 As shown, in this embodiment, the image recognition device may also include a second serial-to-parallel conversion module arranged between the first FPGA unit and the row selection switch, and a third serial-to-parallel conversion module arranged between the first FPGA unit and the first column selection switch. The functions of the second serial-to-parallel conversion module and the third serial-to-parallel conversion module are the same as those of the first serial-to-parallel conversion module, and they will not be described one by one here.

[0041] like Figure 8 As shown, as an implementation mode, in an embodiment of the present application, the second FPGA module includes a second FPGA unit, a first DAC unit, a second DAC unit and an ADC unit, wherein: the first end of the second FPGA unit is electrically connected to the host computer; the second end of the second FPGA module is electrically connected to the first end of the first DAC unit, the first end of the second DAC unit, and the first end of the ADC unit respectively; the second end of the first DAC unit is electrically connected to the input end of the gate selection switch; the second end of the second DAC unit is electrically connected to the input end of the row selection switch; the second end of the ADC unit is electrically connected to the output end of the first column selection switch; the third end of the second FPGA unit is electrically connected to the third end of the first FPGA unit; the first DAC unit is used to convert the digital signal output by the second FPGA unit into an analog signal, and send it to the gate signal end of the memristor array through the gate selection switch; the second DAC unit is used to convert the digital signal output by the second FPGA unit into an analog signal, and send it to the row signal end of the memristor array through the row selection switch; the ADC unit is used to convert the analog signal output by the column signal end of the memristor array through the first column selection switch into a digital signal, and send it to the second FPGA unit.

[0042] In this embodiment, the model of the second FPGA unit can be FPGA XCKU040, DAC (Digital to analog converter), ADC (analog to Digital converter).

[0043] like Figure 8 and Fig.11 As shown, as an implementation mode, in an embodiment of the present application, the second FPGA module also includes a transimpedance amplifier, the transimpedance amplifier includes an amplifier, a feedback resistor component, a first switch component and a second switch component, the feedback resistor component includes a plurality of feedback resistors with different resistance values; the negative input terminal of the transimpedance amplifier is electrically connected to the output terminal of the first column selection switch; the positive input terminal of the transimpedance amplifier is grounded; the output terminal of the transimpedance amplifier is electrically connected to the first terminal of each first switch in the first switch component; the first terminal of each feedback resistor in the feedback resistor component is electrically connected to the negative input terminal of the transimpedance amplifier; the second terminal of each feedback resistor in the feedback resistor component is respectively connected to the second terminal of each first switch in the switch component and the second terminal of each second switch in the second switch component. The first end is electrically connected; the second end of each second switch in the second switch component is electrically connected to the second end of the ADC unit; the second FPGA unit is also used to send a resistance control signal to the first switch component and the second switch component according to a preset amplification factor, so that a first switch in the first switch component and a second switch in the second switch component are closed, wherein the first switch and the second switch are switches electrically connected to the resistance corresponding to the preset amplification factor; the first switch component is used to control a first switch in the first switch component to be closed and the remaining first switches to be disconnected after receiving the resistance control signal; the second switch component is used to control a second switch in the second switch component to be closed and the remaining second switches to be disconnected after receiving the resistance control signal.

[0044] In this embodiment, the preset amplification factor is a pre-set amplification factor, the feedback resistor component may include 4 feedback resistors (R1-R5) with different resistance values, the first switch component includes the first switches S1-S5, and the second switch component includes the second switches S6-S10. If the resistor corresponding to the preset amplification factor is R1, the second FPGA unit sends a resistance control signal to the first switch component and the second switch component according to the preset amplification factor. After receiving the resistance control signal, the first switch component controls the first switch S1 in the first switch component to close and the remaining first switches S2-S5 to open. After receiving the resistance control signal, the second switch component controls the second switch S6 in the second switch component to close and the remaining second switches S7-S10 to open, so that the input signal can be amplified to the preset amplification factor through the transimpedance amplifier. By setting the transimpedance amplifier, the second FPGA unit can adjust the resistance value of the resistor connected to the negative input terminal and the output terminal of the amplifier according to controlling the corresponding switches in the first switch component and the second switch component to close, so as to adaptively adjust the amplification factor of the transimpedance amplifier.

[0045] like Fig.12As shown, the present application also provides an image recognition method, which is applied to an image recognition device, the image recognition device includes a control processing module, a gate selection switch, a row selection switch, a first column selection switch and a memristor array, the memristor array includes a plurality of first memristors arranged in an array, and the method includes: S11, the control processing module obtains the resistance value of multiple second memristors with different resistance values ​​and the resistance value of the second column selection switch, inputs the test voltage to the row signal end of each second memristor, receives the test current corresponding to the resistance value of each second memristor through the second column selection switch, and obtains a current-resistance relationship according to the resistance values ​​of the multiple second memristors, the resistance values ​​of the second column selection switches and the multiple test currents, wherein the multiple second memristors are memristors with known parameters, the second column selection switch is a column selection switch with known parameters, the control processing module is electrically connected to the row signal end of the second memristor and the output end and the control end of the second column selection switch, respectively, and the column signal end of the second memristor is electrically connected to the input end of the second column selection switch; S12, the control processing module converts the pixel value of the image to be recognized to obtain the voltage to be recognized, determines the row and column information of the memristor subarray to be operated in the memristor array according to the image to be recognized, and obtains the weight voltage corresponding to the conductance value of each first memristor according to the conductance value and current-resistance relationship of each first memristor in the memristor subarray, wherein the conductance value of each first memristor is mapped to the weight matrix of the neural network determined according to the image to be recognized; S13, the control processing module controls the gate selection switch, the row selection switch and the switches in the first column selection switch to close based on the row and column information of the memristor subarray, inputs the first gate voltage to the gate signal terminal of the memristor subarray, inputs each weight voltage to the row signal terminal of the memristor subarray accordingly, so that the weights in the weight matrix of the neural network are written into the memristor subarray, inputs the voltage to be identified into each row signal terminal of the memristor subarray, receives each column current outputted from each column signal terminal of the memristor subarray through each first column selection subswitches in the first column selection switch, and determines the recognition result of the image to be identified according to each column current.

[0046] As an implementation manner, in the embodiment of the present application, the method further includes: S21, the control processing module reads the to-be-modulated row and column information in the memristor array in the weight modulation instruction, each target resistance value and each pulse voltage corresponding to the to-be-modulated row and column information, selects the first memristor currently modulated from the memristor array based on the to-be-modulated row and column information, and controls the gate selection switch, the row selection switch and the first column selection switch corresponding to the first memristor currently modulated to close; S22, the control processing module inputs the second gate voltage to the gate signal end of the currently modulated first memristor, inputs the pulse voltage corresponding to the row and column information of the currently modulated first memristor to the row signal end of the currently modulated first memristor, receives the modulated current output by the column signal end of the currently modulated first memristor and then flows through the first column selector switch, obtains the modulated resistance value according to the relationship between the modulated current and the current resistance, determines whether the modulated resistance value is equal to the target resistance value, and if not, adjusts the pulse voltage so that the adjusted modulated resistance value is equal to the target resistance value, until the modulated resistance value of each first memristor corresponding to the row and column information to be modulated is equal to the corresponding target resistance value.

[0047] As an implementation mode, in the embodiment of the present application, the control processing module includes a host computer, a first FPGA unit and a second FPGA module; Step S12 includes: The host computer converts the pixel values ​​of the image to be identified to obtain the voltage to be identified, determines the row and column information of the memristor subarray to be operated in the memristor array according to the image to be identified, obtains the weight voltage corresponding to the conductance value of each first memristor according to the conductance value of each first memristor in the memristor subarray and the current-resistance relationship, generates an image recognition instruction according to the voltage to be identified, the row and column information of the memristor subarray and the weight voltage of each first memristor, and sends it to the first FPGA unit and the second FPGA module, wherein the conductance value of each first memristor is mapped to the weight matrix of the neural network determined according to the image to be identified; Step S13 includes: S131, the first FPGA unit receives an image recognition instruction, reads the row and column information of the memristor subarray in the image recognition instruction, selects the row and column information of the currently operated first memristor from the row and column information of the memristor subarray, sends the row and column information of the currently operated first memristor to the second FPGA module, sends a first gate selection control signal to the gate selection switch, sends a first row selection control signal to the row selection switch, and sends a first column selection control signal to the first column selection switch; S132, the gate selection switch closes a gate selection sub-switch in the gate selection switch according to the first gate selection control signal; S133, the row selection switch closes a row selection sub-switch in the row selection switch according to the first row selection control signal; S134, the first column selection switch closes a first column selection sub-switch in the first column selection switch according to the first column selection control signal; S135, the second FPGA module receives the image recognition instruction, reads the voltage to be recognized, the row and column information of the memristor subarray, and the weight voltage of each first memristor in the image recognition instruction, inputs the first gate voltage to the gate signal terminal of the currently operated first memristor, and inputs the weight voltage corresponding to the row and column information of the currently operated first memristor to the row signal terminal of the currently operated first memristor; S136, after the first FPGA unit writes the corresponding weight voltage to the row signal terminal of each first memristor of the memristor subarray, based on the row and column information of the memristor subarray, sends a second gate selection control signal to the gate selection switch, sends a second row selection control signal to the row selection switch, and sends a second column selection control signal to the first column selection switch; S137, the gate selection switch closes a plurality of gate selection sub-switches in the gate selection switch according to the second gate selection control signal; S138, the row selection switch closes a plurality of row selection sub-switches in the row selection sub-switches according to the second row selection control signal; S139, the first column selection switch closes a plurality of column selection switches in the first column selection switch according to the second column selection control signal; S1310, the second FPGA module inputs the first gate voltage into the gate signal terminal of the memristor subarray, and inputs the voltage to be identified into each row signal terminal of the memristor subarray. The second FPGA module receives each column current output from each column signal terminal of the memristor subarray through each first column selection sub-switches in the first column selection switch, and determines the recognition result of the image to be identified based on each column current, and sends the recognition result to the host computer.

[0048] It should be understood that the use of "system", "device", "unit" and / or "module" in this application is only a method for distinguishing different components, elements, parts, parts or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.

[0049] The embodiments in this specification are described in a progressive manner, and each embodiment focuses on the For the differences between other embodiments, the same or similar parts between the embodiments may be referred to each other.

[0050] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image recognition device, characterized in that: The image recognition device comprises a control processing module, a gate selection switch, a row selection switch, a first column selection switch and a memristor array, wherein the memristor array comprises a plurality of first memristors arranged in an array; The control processing module is electrically connected to the input end and the control end of the gate selection switch, the input end and the control end of the row selection switch, and the output end and the control end of the first column selection switch respectively; The output end of the gate selection switch is electrically connected to the gate signal end of the memristor array; The output end of the row selection switch is electrically connected to the row signal end of the memristor array; An input terminal of the first column selection switch is electrically connected to a column signal terminal of the memristor array; The control processing module is used to obtain the resistance values ​​of multiple second memristors with different resistance values ​​and the resistance value of the second column selection switch, input a test voltage to the row signal terminal of each second memristor, receive a test current corresponding to the resistance value of each second memristor through the second column selection switch, and obtain a current-resistance relationship according to the resistance values ​​of the multiple second memristors, the resistance value of the second column selection switch and the multiple test currents, wherein the multiple second memristors are memristors with known parameters, the second column selection switch is a column selection switch with known parameters, the control processing module is electrically connected to the row signal terminal of the second memristor and the output terminal and control terminal of the second column selection switch, respectively, and the column signal terminal of the second memristor is electrically connected to the input terminal of the second column selection switch; The control processing module is also used to convert the pixel value of the image to be identified to obtain the voltage to be identified. According to the image to be recognized, determining row and column information of a memristor subarray to be operated in the memristor array, and obtaining a weight voltage corresponding to the conductance value of each first memristor in the memristor subarray according to the conductance value of each first memristor and the current-resistance relationship, wherein the conductance value of each first memristor is mapped to a weight matrix of a neural network determined according to the image to be recognized; The control processing module is also used to control the closure of the gate selection switch, the row selection switch and the sub-switches in the first column selection switch based on the row and column information of the memristor sub-array, input the first gate voltage to the gate signal terminal of the memristor sub-array, input each of the weight voltages to the row signal terminal of the memristor sub-array accordingly, so that the weights in the weight matrix of the neural network are written into the memristor sub-array, input the voltage to be identified into each row signal terminal of the memristor sub-array, receive each column current output from each column signal terminal of the memristor sub-array through each first column selection sub-switches in the first column selection switch, and determine the recognition result of the image to be identified based on each of the column currents.

2. The image recognition device according to claim 1, characterized in that: The control processing module is further used to read the row and column information to be modulated in the memristor array in the weight modulation instruction, each target resistance value and each pulse voltage corresponding to the row and column information to be modulated, select the first memristor to be modulated currently from the memristor array based on the row and column information to be modulated, and control the gate selection switch, the row selection switch and the sub-switch corresponding to the first memristor to be modulated currently in the first column selection switch to be closed; The control processing module is also used to input the second gate voltage to the gate signal end of the currently modulated first memristor, input the pulse voltage corresponding to the row and column information of the currently modulated first memristor to the row signal end of the currently modulated first memristor, receive the modulated current after the current output from the column signal end of the currently modulated first memristor flows through the first column selection sub-switch, obtain the modulation resistance value according to the relationship between the modulation current and the current-resistance formula, judge whether the modulation resistance value is equal to the target resistance value, and if not, adjust the pulse voltage so that the adjusted modulation resistance value is equal to the target resistance value, until the modulation resistance value of each of the first memristors corresponding to the row and column information to be modulated is equal to the corresponding target resistance value.

3. The image recognition device according to claim 2, characterized in that: The control processing module includes a host computer, a first FPGA unit and a second FPGA module; The host computer is electrically connected to the first end of the first FPGA unit and the first end of the second FPGA module respectively; The second end of the first FPGA unit is electrically connected to the second end of the second FPGA module; The third terminal of the first FPGA unit is electrically connected to the control terminal of the gate selection switch; The fourth terminal of the first FPGA unit is electrically connected to the control terminal of the row selection switch; The fifth terminal of the first FPGA unit is electrically connected to the control terminal of the first column selection switch; The third terminal of the second FPGA module is electrically connected to the input terminal of the gate selection switch; The fourth terminal of the second FPGA module is electrically connected to the input terminal of the row selection switch; A fifth terminal of the second FPGA module is electrically connected to an output terminal of the first column selection switch; The output end of the gate selection switch is electrically connected to the gate signal end of the memristor array; The output end of the row selection switch is electrically connected to the row signal end of the memristor array; An input terminal of the first column selection switch is electrically connected to a column signal terminal of the memristor array; The host computer is used to convert the pixel values ​​of the image to be identified to obtain the voltage to be identified, determine the row and column information of the memristor subarray to be operated in the memristor array according to the image to be identified, obtain the weight voltage corresponding to the conductance value of each first memristor in the memristor subarray and the current-resistance relationship according to the conductance value of each first memristor in the memristor subarray and the current-resistance relationship, generate an image recognition instruction according to the voltage to be identified, the row and column information of the memristor subarray and the weight voltage of each first memristor, and send it to the first FPGA unit and the second FPGA module, wherein the conductance value of each first memristor is mapped to the weight matrix of the neural network determined according to the image to be identified; The first FPGA unit is used to receive the image recognition instruction, read the row and column information of the memristor subarray in the image recognition instruction, select the row and column information of the currently operated first memristor from the row and column information of the memristor subarray, send the row and column information of the currently operated first memristor to the second FPGA module, send a first gate selection control signal to the gate selection switch, send a first row selection control signal to the row selection switch, and send a first column selection control signal to the first column selection switch; The gate selection switch is used to close a gate selection sub-switch in the gate selection switch according to the first gate selection control signal; The row selection switch is used to close a row selection sub-switch in the row selection switch according to the first row selection control signal; The first column selection switch is used to close a first column selection sub-switch in the first column selection switch according to the first column selection control signal; The second FPGA module is used to receive the image recognition instruction, read the voltage to be recognized, the row and column information of the memristor subarray and the weight voltage of each of the first memristors in the image recognition instruction, input the first gate voltage to the gate signal terminal of the currently operating first memristor, and input the weight voltage corresponding to the row and column information of the currently operating first memristor to the row signal terminal of the currently operating first memristor; The first FPGA unit is further configured to send a second gate selection control signal to the gate selection switch, send a second row selection control signal to the row selection switch, and send a second column selection control signal to the first column selection switch based on the row and column information of the memristor subarray after the corresponding weight voltage is written to the row signal terminal of each first memristor of the memristor subarray; The gate selection switch is further used to close a plurality of gate selection sub-switches in the gate selection switch according to the second gate selection control signal; The row selection switch is further used to close a plurality of row selection sub-switches in the row selection sub-switches according to the second row selection control signal; The first column selection switch is further used to close a plurality of first column selection sub-switches in the first column selection switch according to the second column selection control signal; The second FPGA module is also used to input the first gate voltage into the gate signal terminal of the memristor subarray, input the voltage to be identified into each row signal terminal of the memristor subarray, receive each column current output from each column signal terminal of the memristor subarray through each first column selection sub-switches in the first column selection switch, determine the recognition result of the image to be identified based on each column current, and send the recognition result to the host computer.

4. The image recognition device according to claim 3, characterized in that: The host computer is further used to send a weight modulation instruction, wherein the weight modulation instruction includes the row and column information to be modulated in the memristor array, each target resistance value corresponding to the row and column information to be modulated, and each pulse voltage; The first FPGA unit is used to receive the weight modulation instruction, read the row and column information to be modulated in the weight modulation instruction, select the currently modulated first memristor from the memristor array based on the row and column information to be modulated, and send the corresponding second gate selection control signal to the gate selection switch, send the second row selection control signal to the row selection switch, and send the second column selection control signal to the first column selection switch; The gate selection switch is used to close a gate selection sub-switch in the gate selection switch according to the second gate selection control signal; The row selection switch is used to close a row selection sub-switch in the row selection switch according to the second row selection control signal; The first column selection switch is used to close a first column selection sub-switch in the first column selection switch according to the second column selection control signal; The second FPGA module is used to receive the weight modulation instruction, read the row and column information to be modulated, each target resistance value and each pulse voltage corresponding to the row and column information to be modulated in the weight modulation instruction, input the second gate voltage to the gate signal end of the currently modulated first memristor, input the pulse voltage corresponding to the row and column information of the currently modulated first memristor to the row signal end of the currently modulated first memristor, receive the modulation current after the current output from the column signal end of the currently modulated first memristor flows through the first column selection sub-switch, and send the modulation current to the host computer; The host computer is further used to obtain a modulation resistance value according to the modulation current and the current-resistance relationship, and to determine whether the modulation resistance value is equal to the target resistance value, and if not, to send a voltage control signal to the second FPGA module; The second FPGA module is used to receive the voltage control signal, and based on the voltage control signal, input the adjusted pulse voltage to the row signal end of the currently modulated first memristor, so that the adjusted modulation resistance is equal to the target resistance, until the modulation resistance of each of the first memristors corresponding to the row and column information to be modulated is equal to the corresponding target resistance, wherein the adjusted modulation resistance is obtained based on the adjusted modulation current output from the column signal end of the currently modulated first memristor and the current-resistance relationship.

5. The image recognition device according to claim 3 or 4, characterized in that: It also includes a first serial-to-parallel conversion module, the first serial-to-parallel conversion module includes a shift register and a register, the shift register includes N first flip-flops, the register includes N second flip-flops, wherein N is a positive integer greater than 5, and the gate selection switch is provided with N; The data input terminal of the shift register is electrically connected to the first sub-port in the third terminal of the first FPGA unit; The clock input terminal of each of the first triggers in the shift register is electrically connected to the second sub-port in the third terminal of the first FPGA unit; The reset input terminal of each of the first triggers in the shift register is electrically connected to the third sub-port in the third terminal of the first FPGA unit; The data output terminals of the first flip-flops in the shift register are electrically connected to the data input terminals of the second flip-flops in the register in a one-to-one correspondence; The clock input terminal of each of the second triggers in the register is electrically connected to the fourth subport in the third terminal of the first FPGA unit; The reset input terminal of each of the second flip-flops in the register is electrically connected to the fifth subport in the third terminal of the first FPGA unit; The data output end of each of the second triggers in the register is electrically connected to the control end of the N gate selection switches in a one-to-one correspondence; The first FPGA unit is used to convert an N-bit parallel gate selection control signal into an N-bit serial gate selection control signal and input it into the data input end of the shift register, and then input N first clock pulse signals to the clock input end of each of the first triggers in the shift register, so that the N-bit serial gate selection control signal is shifted and registered in the N second triggers in the register, and after the serial gate selection control signal is shifted and registered in the register, send a second time pulse signal to the clock input end of each of the second triggers in the register, so that the N-bit gate selection control signal in the register is output in parallel to each control end in the gate selection switch, wherein the gate selection control signal includes a first gate selection control signal and / or a second gate selection control signal; The shift register is used to shift and store the N-bit serial gate selection control signal input by the first FPGA unit into the register under the drive of the N first clock pulse signals; The register is used to output the N-bit gate selection control signal after shift registration in parallel to the control ends of the N gate selection switches under the drive of the second clock pulse signal.

6. The image recognition device according to claim 3 or 4, characterized in that: The second FPGA module includes a second FPGA unit, a first DAC unit, a second DAC unit and an ADC unit, wherein: The first end of the second FPGA unit is electrically connected to the host computer; The second end of the second FPGA module is electrically connected to the first end of the first DAC unit, the first end of the second DAC unit, and the first end of the ADC unit respectively; The second end of the first DAC unit is electrically connected to the input end of the gate selection switch; The second end of the second DAC unit is electrically connected to the input end of the row selection switch; The second end of the ADC unit is electrically connected to the output end of the first column selection switch; The third end of the second FPGA unit is electrically connected to the third end of the first FPGA unit; The first DAC unit is used to convert the digital signal output by the second FPGA unit into an analog signal, and send it to the gate signal terminal of the memristor array through the gate selection switch; The second DAC unit is used to convert the digital signal output by the second FPGA unit into an analog signal, and send it to the row signal terminal of the memristor array through the row selection switch; The ADC unit is used to convert the analog signal output by the column signal end of the memristor array through the first column selection switch into a digital signal, and send it to the second FPGA unit.

7. The image recognition device according to claim 6, characterized in that: The second FPGA module further includes a transimpedance amplifier, the transimpedance amplifier includes an amplifier, a feedback resistor component, a first switch component and a second switch component, and the feedback resistor component includes a plurality of feedback resistors with different resistance values; The negative input terminal of the transimpedance amplifier is electrically connected to the output terminal of the first column selection switch; The positive input terminal of the transimpedance amplifier is grounded; The output end of the transimpedance amplifier is electrically connected to the first end of each first switch in the first switch component; A first end of each feedback resistor in the feedback resistor assembly is electrically connected to a negative input terminal of the transimpedance amplifier; The second end of each feedback resistor in the feedback resistor assembly is electrically connected to the second end of each first switch in the switch assembly and the first end of each second switch in the second switch assembly respectively; A second end of each second switch in the second switch assembly is electrically connected to a second end of the ADC unit; The second FPGA unit is further used to send a resistance control signal to the first switch component and the second switch component according to a preset amplification factor, so that a first switch in the first switch component and a second switch in the second switch component are closed, wherein the first switch and the second switch are switches electrically connected to the resistor corresponding to the preset amplification factor; The first switch component is used to control one of the first switches in the first switch component to be closed and the other first switches to be opened after receiving the resistance control signal; The second switch component is used to control one of the second switches in the second switch component to be closed and the other second switches to be opened after receiving the resistance control signal.

8. An image recognition method, characterized in that: Applied to an image recognition device, the image recognition device includes a control processing module, a gate selection switch, a row selection switch, a first column selection switch and a memristor array, the memristor array includes a plurality of first memristors arranged in an array, and the method includes: The control processing module obtains the resistance values ​​of multiple second memristors with different resistance values ​​and the resistance values ​​of the second column selection switches, inputs a test voltage to the row signal terminal of each of the second memristors, receives a test current corresponding to the resistance value of each of the second memristors through the second column selection switch, and obtains a current-resistance relationship according to the resistance values ​​of the multiple second memristors, the resistance values ​​of the second column selection switches and the multiple test currents, wherein the multiple second memristors are memristors with known parameters, the second column selection switch is a column selection switch with known parameters, the control processing module is electrically connected to the row signal terminal of the second memristor and the output terminal of the second column selection switch, respectively, and the column signal terminal of the second memristor is electrically connected to the input terminal of the second column selection switch; The control processing module converts the pixel values ​​of the image to be identified to obtain the voltage to be identified, determines the row and column information of the memristor subarray to be operated in the memristor array according to the image to be identified, and obtains the weight voltage corresponding to the conductance value of each first memristor in the memristor subarray according to the conductance value of each first memristor and the current-resistance relationship, wherein the conductance value of each first memristor is mapped to the weight matrix of the neural network determined according to the image to be identified; The control processing module controls the gate selection switch, the row selection switch and the sub-switches in the first column selection switch to close based on the row and column information of the memristor subarray, inputs the first gate voltage to the gate signal terminal of the memristor subarray, inputs each of the weight voltages to the row signal terminal of the memristor subarray accordingly, so that the weights in the weight matrix of the neural network are written into the memristor subarray, inputs the voltage to be identified into each row signal terminal of the memristor subarray, receives each column current outputted from each column signal terminal of the memristor subarray through each of the first column selection sub-switches in the first column selection switch, and determines the recognition result of the image to be identified based on each of the column currents.

9. The image recognition method according to claim 8, characterized in that: The method further comprises: The control processing module reads the to-be-modulated row and column information in the memristor array in the weight modulation instruction, each target resistance value and each pulse voltage corresponding to the to-be-modulated row and column information, selects the currently modulated first memristor from the memristor array based on the to-be-modulated row and column information, and controls the gate selection switch, the row selection switch and the sub-switch in the first column selection switch corresponding to the currently modulated first memristor to be closed; The control processing module inputs the second gate voltage to the gate signal end of the currently modulated first memristor, inputs the pulse voltage corresponding to the row and column information of the currently modulated first memristor to the row signal end of the currently modulated first memristor, receives the modulated current output by the column signal end of the currently modulated first memristor and then flows through the first column selection sub-switch, obtains the modulation resistance value according to the relationship between the modulation current and the current-resistance formula, determines whether the modulation resistance value is equal to the target resistance value, and if not, adjusts the pulse voltage so that the adjusted modulation resistance value is equal to the target resistance value, until the modulation resistance value of each of the first memristors corresponding to the row and column information to be modulated is equal to the corresponding target resistance value.

10. The image recognition method according to claim 9, characterized in that: The control processing module includes a host computer, a first FPGA unit and a second FPGA module; The control processing module converts the pixel value of the image to be recognized to obtain the voltage to be recognized, determines the row and column information of the memristor subarray to be operated in the memristor array according to the image to be recognized, and obtains the weight voltage corresponding to the conductance value of each first memristor in the memristor subarray according to the conductance value of each first memristor and the current-resistance relationship, wherein the conductance value of each first memristor is mapped to the weight matrix of the neural network determined according to the image to be recognized, including: The host computer converts the pixel values ​​of the image to be identified to obtain the voltage to be identified, determines the row and column information of the memristor subarray to be operated in the memristor array according to the image to be identified, obtains the weight voltage corresponding to the conductance value of each first memristor according to the conductance value of each first memristor in the memristor subarray and the current-resistance relationship, generates an image recognition instruction according to the voltage to be identified, the row and column information of the memristor subarray and the weight voltage of each first memristor, and sends it to the first FPGA unit and the second FPGA module, wherein the conductance value of each first memristor is mapped to the weight matrix of the neural network determined according to the image to be identified; The control processing module controls the gate selection switch, the row selection switch and the sub-switches in the first column selection switch to close based on the row and column information of the memristor subarray, inputs the first gate voltage to the gate signal terminal of the memristor subarray, and inputs each of the weight voltages to the row signal terminal of the memristor subarray accordingly, so that the weights in the weight matrix of the neural network are written into the memristor subarray, inputs the voltage to be identified to each row signal terminal of the memristor subarray, receives each column current outputted from each column signal terminal of the memristor subarray through each first column selection sub-switches in the first column selection switch, and determines the recognition result of the image to be identified according to each of the column currents, including: The first FPGA unit receives the image recognition instruction, reads the row and column information of the memristor subarray in the image recognition instruction, selects the row and column information of the currently operated first memristor from the row and column information of the memristor subarray, sends the row and column information of the currently operated first memristor to the second FPGA module, sends a first gate selection control signal to the gate selection switch, sends a first row selection control signal to the row selection switch, and sends a first column selection control signal to the first column selection switch; The gate selection switch closes a gate selection sub-switch in the gate selection switch according to the first gate selection control signal; The row selection switch closes a row selection sub-switch in the row selection switch according to the first row selection control signal; The first column selection switch closes a first column selection sub-switch in the first column selection switch according to the first column selection control signal; The second FPGA module receives the image recognition instruction, reads the voltage to be recognized, the row and column information of the memristor subarray, and the weight voltage of each of the first memristors in the image recognition instruction, inputs the first gate voltage to the gate signal terminal of the currently operating first memristor, and inputs the weight voltage corresponding to the row and column information of the currently operating first memristor to the row signal terminal of the currently operating first memristor; After the row signal end of each first memristor of the memristor subarray writes the corresponding weight voltage, the first FPGA unit sends a second gate selection control signal to the gate selection switch based on the row and column information of the memristor subarray, sends a second row selection control signal to the row selection switch, and sends a second column selection control signal to the first column selection switch; The gate selection switch closes a plurality of gate selection sub-switches in the gate selection switch according to the second gate selection control signal; The row selection switch closes a plurality of row selection sub-switches in the row selection sub-switches according to the second row selection control signal; The first column selection switch closes a plurality of column selection switches in the first column selection switch according to the second column selection control signal; The second FPGA module inputs the first gate voltage into the gate signal terminal of the memristor subarray, inputs the voltage to be identified into each row signal terminal of the memristor subarray, receives each column current output from each column signal terminal of the memristor subarray through each first column selection sub-switches in the first column selection switch, determines the recognition result of the image to be identified based on each column current, and sends the recognition result to the host computer.

Images