High-precision detection and recognition system and method for integrated sensing and computing targets

Abstract

The present invention provides a high-precision detection and identification system and method for integrated sensing and computing targets, comprising: constructing a target detection network using a memristor array and applicable peripheral circuits; employing a complex teacher network to guide the feature extraction process of the target detection network; obtaining memristor resistance drift and instability parameters from an integrated sensing, storage, and computing chip, modeling memristor available resistance states, Gaussian, and salt-and-pepper noise functions, and constructing a memristor non-ideal characteristic model; using the memristor non-ideal characteristic model to convert the memristor non-ideal characteristics into superimposed noise of target detection network weights; generating new weight values ​​using the superimposed noise, converting the new weight values ​​into memristor resistance values, and constructing a sensing and computing integrated target detection network; and utilizing the sensing and computing integrated target detection network to detect and identify a target to be detected from an input signal. The present invention solves the technical problem that the non-ideal characteristics of memristors lead to perturbations in memristor weights, poor weight representation capability, and low target detection and identification accuracy.

Description

Technical Field

[0001] The present invention relates to the field of intelligent perception and image recognition technology, and specifically to a high-precision detection and recognition system and method for integrated sensing and computing targets. Background Art

[0002] The target detection and recognition system based on integrated sensing, storage and computing can achieve target detection and recognition tasks at ultra-high speed. It uses memristors to simulate the weights of the neural network in the traditional deep learning network implemented on the computer, and implements the multiplication and accumulation operations of the convolution process based on Kirchhoff's law. The entire neural network is implemented with hardware circuits. Therefore, the target detection and recognition system based on integrated sensing, storage and computing breaks through the traditional architecture of separation of acquisition, storage and calculation, and has the advantage of ultra-high-speed computing.

[0003] The existing invention patent application document, "Circuit System for Memristor Array Weight Modulation and Image Recognition," with publication number CN115840527A, describes a circuit system comprising a PC, an FPGA chip, a digital-to-analog conversion unit, a switch unit, a memristor array unit, an integration and signal amplification circuit, and an analog-to-digital converter. The invention uses PC-specific software to select the required functions, which can be array reading and writing, weight modulation, and image recognition. Commands or image RGB values ​​collected by the PC are converted into corresponding grayscale values ​​and sent to the FPGA chip. The FPGA chip then controls the selection of the memristor array unit to be modulated through the digital-to-analog conversion unit and the switch unit. The existing invention patent application document with publication number CN110443168A, "A neural network face recognition system based on memristors," includes a face capture module, a preprocessing module, an input module, a memristor neural network module, an output module, and a weight update module; the face capture module is used to capture a face image in a picture; the preprocessing module is used to reduce the dimension of the face image; the input module is used to convert the reduced-dimensional image into an electrical signal; the memristor neural network module is used to store network weights and perform matrix-vector multiplication on the electrical signal, and the operation result is transmitted to the output module; the output module transmits the operation result to the weight update module for weight update, and the updated weight is transmitted to the memristor neural network module, and the output module reads the recognition result of the network; the memristor neural network module is composed of a memristor array. Although the aforementioned prior art solution based on the sensing, storage, and computing integrated chip of memristors has the above advantages, the current memristors still have non-ideal characteristics such as resistance drift and resistance state instability, which causes inaccurate modulation when the offline trained neural network weights are mapped to the resistance value of the corresponding memristor in the sensing, storage, and computing integrated chip, thereby reducing the network detection accuracy. Therefore, it is necessary to integrate the non-ideal characteristics of memristors into the generation process of neural network weights during offline training of neural networks.

[0004] In summary, the existing technology has technical problems such as the non-ideal characteristics of the memristor leading to perturbations in the memristor weights and poor weight representation capabilities, which results in low target detection and recognition accuracy. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: how to solve the technical problem in the prior art that the non-ideal characteristics of the memristor lead to perturbation of the memristor weights, poor weight representation ability, and low target detection and recognition accuracy.

[0006] The present invention solves the above technical problems by adopting the following technical solutions: A high-precision target detection and recognition system integrating sensing and computing includes:

[0007] A memristor neural network circuit implementation module for an integrated sensing, storage, and computing chip, which is used to construct a target detection network using a memristor array and applicable peripheral circuits;

[0008] The target detection network knowledge distillation module uses a complex teacher network to guide the feature extraction process of the target detection network during training. The target detection network knowledge distillation module is connected to the memristor neural network circuit of the integrated sensing, storage and computing chip to achieve module connection;

[0009] The memristor non-ideal characteristic modeling module is used to obtain the resistance drift parameters and instability parameters of the memristor from the integrated sensing, storage and computing chip, and use them to model the memristor's available resistance state, Gaussian noise function, and salt and pepper noise function, thereby constructing a memristor non-ideal characteristic model;

[0010] The target detection network weight fault tolerance module is used to utilize the non-ideal characteristic model of the memristor and convert the non-ideal characteristics of the memristor into superimposed noise on the target detection network weights;

[0011] The target detection network weight value to on-chip memristor resistance mapping module is used to generate new weight values ​​using superimposed noise, and convert the new weight values ​​into memristor resistance values ​​within the integrated sensing, storage, and computing chip, thereby constructing the integrated sensing and computing target detection network. The target detection network weight value to on-chip memristor resistance mapping module is connected to the target detection network weight fault tolerance module;

[0012] The target detection network module is used to detect and identify the target to be detected from the input signal using the integrated sensing and computing target detection network. The target detection network module is connected to the target detection network weight value to the on-chip memristor resistance mapping module and the sensing, storage and computing integrated chip memristor neural network circuit to achieve module connection.

[0013] The present invention improves the system detection accuracy from two aspects: improving the weight representation ability during the training process and overcoming the influence of the non-ideal characteristics of the memristor. It has better fault tolerance and higher detection accuracy. After the neural network weights are mapped to the sensor-storage-computing integrated chip, the disturbance of the memristor weights caused by the non-ideal characteristics of the memristor can be tolerated. The present invention has lower power consumption and faster calculation time. Compared with the implementation method based on NVIDIA graphics card on the computer, the processing time of the neural network structure in the hardware implementation method of the sensor-storage-computing integrated chip is the circuit response time, which can basically achieve instantaneous response. At the same time, the hardware simulation circuit does not require NVIDIA graphics card and CPU, and has lower power consumption.

[0014] In a more specific technical solution, the target detection network module includes: a feature encoding module, a feature decoding module, a category recognition module and a position prediction module;

[0015] The feature encoding module uses the ResNet18 backbone network to encode the light intensity information irradiated on the photoelectric memristor array to obtain light intensity encoded output information;

[0016] A feature decoding module is used to decode and process the light intensity encoded output information to obtain a light intensity decoding feature map. The feature decoding module is connected to the feature encoding module.

[0017] A category recognition module is used to decode the feature map based on the light intensity, obtain and output a preset dimension vector, wherein the dimension of the preset dimension vector is adapted to the number of target categories to be detected; the category recognition module is connected to the feature decoding module;

[0018] The position prediction module is used to perform position prediction based on the light intensity decoding feature map to obtain and output a Gaussian heat map, with the maximum value point of the Gaussian heat map as the target center position. The position prediction module is connected to the feature decoding module.

[0019] In a more specific technical solution, the sensing, storage and computing integrated chip memristor neural network circuit implementation module includes: a photoelectric memristor hardware perception module, a current signal conversion module to voltage signal module, an electrical state memristor feature extraction module and a recognition result output module;

[0020] Photoelectric memristor hardware sensing module, used to sense and output current signal I according to the input light signal i ; Includes a photoelectric sensor array to correspond to the first convolutional layer of the target detection network;

[0021] The current signal to voltage signal module is used to convert the current signal I i , through the T-type structure and preset resistors, gain operation is performed to obtain and output voltage signals. The current signal to voltage signal module is connected to the photoelectric memristor hardware sensing module;

[0022] The electrical state memristor feature extraction module is used to process the current signal I based on Kirchhoff's law with preset extraction logic i , based on which the electrical state memristor features are extracted and the target detection network is used to perform weighted multiplication and accumulation operations:

[0023] f(x, W) = W·x + b

[0024] Where W represents the weight value of the target detection network, x is the network input, b is the bias, and the memristor voltage is multiplied by the conductance value to perform a convolution operation and cumulative summation operation. The current of the transformed parallel circuit is added to obtain the output current signal I out , as the electrical memristor characteristic:

[0025] I out =I1+I2+…+I n =U1·G1+U2·G2+…+U n ·G n ;

[0026] Among them, U1,…,U n is the voltage value corresponding to the input x, G1,…,G n is the memristor conductance value corresponding to the weight W, I1,…,I n is the output result of the convolution operation;

[0027] The recognition result output module is used to perform AD sampling on the electric state memristor characteristics to obtain and output the target recognition result. The recognition result output module is connected to the electric state memristor characteristic extraction module.

[0028] In a more specific technical solution, the current signal to voltage signal module includes: an operational amplifier, a first resistor, a second resistor, and a third resistor. The current signal to voltage signal module uses the following logic processing to obtain the voltage signal V0:

[0029] V0=-I i (1+R2 / R3)·R1

[0030] The first resistor R1, the second resistor R2 and the third resistor R3 form a T-shaped structure. The first end of the third resistor R3 is connected to the first resistor R1 and the second resistor R2. The second end of the third resistor R3 is grounded. The current signal I i They are respectively connected to the negative terminal of the operational amplifier and the first resistor R1, the positive terminal of the operational amplifier is grounded, and the second resistor R2 is connected to the output terminal of the operational amplifier.

[0031] In a more specific technical solution, the target detection network knowledge distillation module includes: a complex teacher network and a distillation loss function design module:

[0032] The complex teacher network uses the ResNet101 backbone network. The complex teacher network includes: no less than 2 complex network convolution modules ConvBlock and no less than 2 downsampling layers MaxPooling; the complex network convolution module ConvBlock includes: Conv2D convolution and complex network activation layer Relu;

[0033] The distillation loss function design module is used to obtain and calculate the knowledge distillation loss KD based on the KL divergence:

[0034]

[0035] Among them, p T is the probability predicted by the complex teacher network, p S is the probability predicted by the target detection network, represents the probability predicted by the complex teacher network at pixel t, represents the probability predicted by the target detection network at pixel t, C is the predicted category, T represents the teacher network, and S represents the target detection network designed in this project;

[0036] The present invention has higher detection accuracy. During the weight training process, the present invention adopts a more complex network to guide the feature extraction process of the target detection network. Generally, within a certain range, the increase in the complexity of the network structure is conducive to improving its detection ability. Through the distillation method, a more complex network is used to guide the training process of the target detection network, which can improve the feature expression ability of the target detection network and thus improve the detection accuracy of the target detection network.

[0037] In a more specific technical solution, the memristor non-ideal characteristic modeling module includes: a resistance state value reading and writing data module, a write error modeling module, and a memristor bad point modeling module;

[0038] The resistance state value reading and writing data module is used to represent the available resistance state R of the memristor using the following logic:

[0039] R={R1,R2,…,R N}

[0040] Where N is the number of available resistance states. Based on the available resistance states of the memristor, we repeatedly write into any resistance state and perform a preset number of measurements to record the exact number of times k the resistance value is written into each resistance state. The following logic is then used to obtain the resistance state write accuracy parameter Acc:

[0041] Acc={k1,k2,…,k N}

[0042] A write error modeling module is used to model the write accuracy parameter as a Gaussian noise function P(x), and the write error modeling module is connected to the resistive state resistance value reading and writing data module;

[0043] The memristor bad point modeling module is used to count the damage of the memristors in the memristor array to obtain the distribution of the memristor bad points, so as to obtain and model the probability of the memristor bad points as the salt and pepper noise function B(x).

[0044] The present invention uses a memristor non-ideal characteristic modeling module to construct a mathematical model for the non-ideal characteristics of the memristor contained in the integrated sensing, storage and computing chip, such as resistance drift and resistance state instability; and integrates the non-ideal characteristics of the memristor into the training process of the target detection network, thereby overcoming the influence of the non-ideal characteristics of the memristor on the network detection accuracy.

[0045] In a more specific technical solution, the target detection network weight fault tolerance module includes: a detection network weight extraction module, a memristor noise addition module, a weight value writing module, a weight update module, and a loop update module;

[0046] The detection network weight extraction module is used to obtain all the initial network weights Weights(x) of the target detection network;

[0047] Memristor noise addition module is used to add Gaussian noise function P(x) and salt and pepper noise function B(x) to the initial network weight Weights(x) to obtain the new weight value Weights New (x), the memristor noise adding module 52 is connected to the detection network weight taking module 51:

[0048] Weights New (x) = Weights(x) + P(x) + B(x);

[0049] Weight values ​​are written into the module to write new weight values New (x) Writing into each neuron of the target detection network, the weight value writing module is connected to the memristor noise adding module;

[0050] Weight update module, used to input training data and update the weights according to the new weight values New Perform forward propagation of the target detection network, calculate the gradient information based on the preset loss function, and update the new weight value accordingly New ;

[0051] Loop update module, repeatedly update the new weight value Weights New , based on which the applicable weights Weights_Final are obtained, and the loop update module is connected to the weight update module.

[0052] The present invention quantifies the non-ideal characteristics of the memristor into a noise model and randomly adds the noise to the network weights during the training process, thereby integrating the non-ideal characteristics of the memristor into the neural network during the training process. The network has acquired the ability to resist weight perturbations. After the neural network weights are mapped to the integrated sensing, storage and computing chip, the perturbations of the memristor weights caused by the non-ideal characteristics of the memristor can be tolerated, thus having better fault tolerance.

[0053] In a more specific technical solution, the target detection network weight value to the on-chip memristor resistance mapping module includes: a weight maximum value processing module, a memristor resistance writing module, a detection network weight value mapping module, and a memristor corresponding resistance writing module;

[0054] The weight maximum value processing module is used to extract the applicable weights Weights_Final and calculate the maximum weight value weight_max and the minimum weight_min;

[0055] The memristor resistance writing module is used to calculate the maximum value R_max and the minimum value R_min that can be written to the memristor resistance, and then process the maximum conductance value G of the memristor. max =1 / R_min and the minimum conductance of the memristor G min =1 / R_max;

[0056] The detection network weight value mapping module is used to use the following logic to determine the maximum conductance value G of the memristor. max =1 / R_min and the minimum conductance of the memristor G min =1 / R_max, mapping the applicable network weight weights_Final(i) to the memristor conductance value G(i), and connecting the detection network weight value mapping module with the memristor resistance value writing module and the weight maximum value processing module;

[0057] The memristor corresponding resistance writing module is used to convert all memristor conductance values ​​G(i) into memristor resistance values ​​R(i), and write the memristor resistance values ​​R(i) into the corresponding memristors in the integrated sensing, storage and computing chip.

[0058] In a more specific technical solution, the detection network weight value mapping module uses the following logic to map the applicable network weight weights_Final(i) to the memristor conductance value G(i):

[0059] G(i)=(G max -G min )·(weights_Final(i)-weight-min) / (weight_max-weitht_min).

[0060] In a more specific technical solution, the high-precision detection and recognition method for integrated sensing and computing targets includes:

[0061] S1. Build a target detection network using a memristor array and applicable peripheral circuits.

[0062] S2. During the training process, a complex teacher network is used to guide the feature extraction process of the target detection network;

[0063] S3. Obtain the resistance drift parameters and instability parameters of the memristor from the integrated sensing, storage, and computing chip, and use them to model the memristor's available resistance states, Gaussian noise function, and salt and pepper noise function, thereby constructing a non-ideal characteristic model of the memristor.

[0064] S4. Using the non-ideal characteristic model of the memristor, the non-ideal characteristics of the memristor are converted into superimposed noise on the weights of the target detection network;

[0065] S5. Generate new weight values ​​using superimposed noise, convert the new weight values ​​into memristor resistance values ​​within the integrated sensing, storage, and computing chip, and use them to construct an integrated sensing and computing target detection network.

[0066] S6. Use the sensing and computing integrated target detection network to detect and identify the target to be detected from the input signal.

[0067] Compared with the existing technology, the present invention has the following advantages: the present invention improves the system detection accuracy from two aspects: improving the weight representation ability during the training process and overcoming the influence of the non-ideal characteristics of the memristor by the weight, has better fault tolerance and higher detection accuracy, and can tolerate the disturbance of the memristor weight caused by the non-ideal characteristics of the memristor after the neural network weight is mapped to the integrated sensing, storage and computing chip. The present invention has lower power consumption and faster computing time. Compared with the implementation method based on NVIDIA graphics card on the computer, the processing time of the neural network structure in the hardware implementation method of the integrated sensing, storage and computing chip is the circuit response time, which can basically achieve instantaneous response. At the same time, the hardware simulation circuit does not require NVIDIA graphics card and CPU, and has lower power consumption.

[0068] The present invention has higher detection accuracy. During the weight training process, the present invention adopts a more complex network to guide the feature extraction process of the target detection network. Generally, within a certain range, the increase in the complexity of the network structure is conducive to improving its detection ability. Through the distillation method, a more complex network is used to guide the training process of the target detection network, which can improve the feature expression ability of the target detection network and thus improve the detection accuracy of the target detection network.

[0069] The present invention uses a memristor non-ideal characteristic modeling module to construct a mathematical model for the non-ideal characteristics of the memristor contained in the integrated sensing, storage and computing chip, such as resistance drift and resistance state instability; and integrates the non-ideal characteristics of the memristor into the training process of the target detection network, thereby overcoming the influence of the non-ideal characteristics of the memristor on the network detection accuracy.

[0070] The present invention quantifies the non-ideal characteristics of the memristor into a noise model and randomly adds the noise to the network weights during the training process, thereby integrating the non-ideal characteristics of the memristor into the neural network during the training process. The network has acquired the ability to resist weight perturbations. After the neural network weights are mapped to the integrated sensing, storage and computing chip, the perturbations of the memristor weights caused by the non-ideal characteristics of the memristor can be tolerated, thus having better fault tolerance.

[0071] The present invention solves the technical problem in the prior art that the non-ideal characteristics of the memristor lead to memristor weight disturbance, poor weight representation ability, and low target detection and recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is a schematic diagram of data flow processing in the high-precision sensing and computing integrated target detection and recognition system according to Example 1 of the present invention;

[0073] Figure 2 This is a schematic diagram showing the connection of basic modules of the sensing and computing integrated target high-precision detection and recognition system according to Example 1 of the present invention;

[0074] Figure 3 This is a schematic diagram of the structure of the sensing, storage and computing integrated target detection network module of Example 1 of the present invention;

[0075] Figure 4 This is a schematic diagram of the module structure of the memristor neural network circuit implementation module of the sensing, storage and computing integrated chip according to Example 1 of the present invention;

[0076] Figure 5 Schematic diagram of the specific steps of the non-ideal characteristics modeling process of the memristor according to Example 2 of the present invention;

[0077] Figure 6 This is a schematic diagram of the specific steps of the target detection network weight update process according to embodiment 2 of the present invention;

[0078] Figure 7 Schematic diagram of the mapping process from the weight value of the detection network to the resistance value of the memristor on the chip in Example 2 of the present invention. DETAILED DESCRIPTION

[0079] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0080] Example 1

[0081] like Figure 1 and Figure 2 As shown, the sensing and computing integrated target high-precision detection and identification system provided by the present invention includes: a target detection network module 1, a sensing, storage and computing integrated chip memristor neural network circuit implementation module 2, a target detection network knowledge distillation module 3, a memristor non-ideal characteristic modeling module 4, a target detection network weight fault tolerance module 5, and a target detection network weight value to chip memristor resistance mapping module 6.

[0082] In this embodiment, the target detection network module 1 adopts a convolutional neural network structure, and uses the target detection network module 1 to detect and identify the target to be detected from the input signal.

[0083] like Figure 3 As shown, in this embodiment, the target detection network module 1 includes: a feature encoding module 11, a feature decoding module 12, a category recognition module 13 and a position prediction module 14.

[0084] In this embodiment, the feature encoding module 11 uses the ResNet18 backbone network. In this embodiment, the feature encoding module 11 includes, but is not limited to, four ConvBlocks and four downsampling MaxPooling layers. In this embodiment, each ConvBlock includes one Conv2D convolution and one ReLU, with no normalization layer. The input of the feature encoding module 11 is the light intensity information irradiated on the photoelectric memristor array.

[0085] In this embodiment, feature decoding module 12 uses the output of feature encoding module 11 as input. In this embodiment, feature decoding module 12 includes, but is not limited to, four ConvBlocks and four upsampling layers. In this embodiment, each ConvBlock contains one Conv2D convolution, one activation layer (Relu), and no normalization layer. Feature encoding module 11 is connected to feature encoding module 11.

[0086] In this embodiment, the category recognition module 13 uses the feature map obtained by the feature decoding module 12 as input. In this embodiment, the category recognition module 13 includes, but is not limited to, one global pooling layer, two fully connected Liner layers, and one Relu activation layer. In this embodiment, the output of the category recognition module 13 is a one-dimensional vector whose dimension corresponds to the number of categories of the detected object. The category recognition module 13 is connected to the feature decoding module 12.

[0087] In this embodiment, the position prediction module 14 uses the feature map obtained by the feature decoding module 12 as input. In this embodiment, the position prediction module 14 includes, but is not limited to, two convolutional layers (Conv2D) and one activation layer (Relu). In this embodiment, the output of the position prediction module 14 is a Gaussian heatmap, where the maximum point of the Gaussian heatmap is the location of the target center. The position prediction module 14 is connected to the feature decoding module 12.

[0088] In this embodiment, the sensing-storage-computing integrated chip memristor neural network circuit implementation module 2 includes: a memristor array and corresponding peripheral circuits, and the sensing-storage-computing integrated chip memristor neural network circuit implementation module 2 hardware implements the target detection network;

[0089] In this embodiment, the sensing, storage and computing integrated chip memristor neural network circuit implementation module 2 includes: a photoelectric memristor hardware perception module 21, a current signal conversion voltage signal module 22, an electrical state memristor feature extraction module 23 and a recognition result output module 24.

[0090] In this embodiment, the photoelectric memristor hardware perception module 21 includes a photosensor array with modulatable light responsivity, corresponds to the first convolutional layer of the target detection network, and uses the optical signal as the input of the photosensor array.

[0091] like Figure 4 As shown, in this embodiment, the current signal to voltage signal module 22 includes: an operational amplifier and a resistor. In this embodiment, the weak current signal I output by the memristor array i , a larger gain multiple is achieved by using a smaller resistor through a T-type structure, and the output voltage is:

[0092] V0=-I i (1+R2 / R3)·R1

[0093] Among them, R1, R2, and R3 form a T-type structure. One end of R3 is connected to R1 and R2, and one end is grounded. The current I i They are respectively connected to the negative terminal of the operational amplifier and one end of R1, the positive pole of the operational amplifier is grounded, R2 is connected to the output end of the operational amplifier, and the current signal to voltage signal module 22 is connected to the photoelectric memristor hardware sensing module 21.

[0094] In this embodiment, the electrical state memristor feature extraction module 23 includes a memristor array. In this embodiment, the electrical state memristor feature extraction module 23 is based on Kirchhoff's law:

[0095]

[0096] Where U is the voltage applied across the memristor, R is the resistance of the memristor, and G is the conductance of the memristor. The multiplication and accumulation operation of the input and weight from the second layer of the target detection network to the output neuron at the end of the network is:

[0097] f(x, W) = W·x + b

[0098] The convolution operation is converted into the form of multiplying the memristor voltage and its conductance. The cumulative summation operation is converted into the current addition of the parallel circuit. The electrical state memristor feature extraction module 23 is connected to the current signal conversion module 22:

[0099] I out =I1+I2+…+I n =U1·G1+U2·G2+…+U n ·G n ;

[0100] In this embodiment, the recognition result output module 24 includes a current-to-voltage circuit and an AD sampling circuit. In this embodiment, the voltage range of AD sampling includes but is not limited to: 0-0.2v, and the sampling accuracy can be, for example: 8bit; the recognition result output module 24 is connected to the electrical state memristor feature extraction module 23.

[0101] In this embodiment, the target detection network knowledge distillation module 3 is used to improve the feature expression ability of the target detection network. During the training process, a complex network is used to guide the feature extraction process of the target detection network.

[0102] In this embodiment, the target detection network knowledge distillation module 3 includes: a complex teacher network 31 and a distillation loss function design module 32.

[0103] In this embodiment, the complex teacher network 31 utilizes the ResNet101 backbone. This network includes, but is not limited to, four ConvBlocks and four MaxPooling downsampling layers. Each ConvBlock includes, but is not limited to, one Conv2D convolution and one ReLU, with no normalization layer. The input to the feature encoding module 11 is the light intensity information incident on the photoelectric memristor array.

[0104] In this embodiment, the distillation loss function design module 32 uses KL divergence to calculate the knowledge distillation loss:

[0105]

[0106] In this embodiment, the memristor non-ideal characteristic modeling module 4 is used to construct a mathematical model for the non-ideal characteristics of the memristor included in the sensing, storage and computing integrated chip, such as resistance drift and instability.

[0107] In this embodiment, in the memristor non-ideal characteristic modeling module 4, the available resistance states of the memristor are expressed as:

[0108] R={R1,R2,…,R N}

[0109] Where N is the number of available resistance states.

[0110] In this embodiment, the memristor non-ideal characteristic modeling module 4 includes:

[0111] The resistance reading and writing module 41 repeatedly writes and measures any resistance state K times, and records the accurate number of times each resistance state is written as Acc = {k1, k2, ..., k N}, the writing accuracy of the resistive state will be represented by Acc.

[0112] The write error modeling module 42 is used to model the statistical write error as a Gaussian noise function P(x) based on the read and write data. The write error modeling module 42 is connected to the resistance value reading and writing data module 41.

[0113] The memristor bad point modeling module 43 is used to collect statistics on the damage of memristors in the memristor array and model the probability of bad points in the memristor as a salt and pepper noise function B(x) based on the bad point distribution.

[0114] In this embodiment, the target detection network weight fault tolerance module 5 is used to integrate the non-ideal characteristics of the memristor into the training process of the target detection network, thereby overcoming the impact of the non-ideal characteristics of the memristor on the network detection accuracy. The memristor non-ideal characteristic function constructed by the memristor non-ideal characteristic modeling module 4 is used to convert the non-ideal characteristics of the memristor into noise superimposed on the target detection network weights during the training process.

[0115] In this embodiment, the target detection network weight fault tolerance module 5 includes:

[0116] The detection network weight extraction module 51 extracts all weights Weights(x) of the target detection network before the forward propagation of the target detection network training phase begins;

[0117] The memristor noise adding module 52 is used to add the memristor non-ideal characteristic noise P(x) and bad pixel noise B(x) to the weight Weights(x). The memristor noise adding module 52 is connected to the detection network weight extraction module 51:

[0118] Weights New (x) = Weights(x) + P(x) + B(x);

[0119] The weight value is written into module 53, and the new weight value is New (x) Writing into each neuron of the target detection network, the weight value writing module 53 is connected to the memristor noise adding module 52;

[0120] The weight update module 54 inputs the training data and updates the weights New Complete the forward propagation, calculate the gradient information based on the loss function, and update the Weights New , the weight updating module 54 is connected to the weight value writing module 53;

[0121] A cyclic updating module 55 is configured to repeatedly update the weight value, and the cyclic updating module 55 is connected to the weight updating module 54;

[0122] In this embodiment, a target detection network weight value to chip memristor resistance mapping module 6 is used to convert the neuron weight values ​​of the target detection network trained on a computer into the resistance values ​​of the memristors in the sensing, storage and computing integrated chip, thereby constructing a target detection network based on memristor array hardware.

[0123] In this embodiment, the target detection network weight value to the on-chip memristor resistance mapping module 6 includes:

[0124] The weight maximum processing module 61 takes out the weights Weights_Final completed by the target detection network training and calculates the maximum value weight_max and the minimum value weight_min of the weights;

[0125] The memristor resistance writing module 62 calculates the maximum value R_max and the minimum value R_min of the memristor resistance that can be written. The maximum conductance value G that can be written to the memristor is max =1 / R_min, the minimum conductivity is G min =1 / R_max;

[0126] The detection network weight value mapping module 63 is used to map the target detection network weight value weights_Final(i) to the conductance value G(i) of the memristor in the integrated sensing, storage and computing chip, as follows:

[0127] G(i)=(Gmax -G min )·(weights_Final(i)−weight_min) / (weight_max−weitht_min). The detection network weight value mapping module 63 is connected to the memristor resistance value writing module 62 and the weight maximum value processing module 61 .

[0128] The resistance value writing module 64 corresponding to the memristor converts all conductance values ​​G(i) into the resistance value R(i) = 1 / G(i) of the memristor, and writes the resistance value R(i) into the corresponding memristor in the integrated sensing, storage and computing chip. The resistance value writing module 64 corresponding to the memristor is connected to the detection network weight value mapping module 63.

[0129] Example 2

[0130] In this embodiment, the high-precision detection and identification method for integrated sensing and computing targets provided by the present invention includes: a non-ideal characteristic modeling process of a memristor, a target detection network weight update process, and a target detection network weight value to a memristor resistance value mapping process within the chip.

[0131] In this embodiment, the non-ideal characteristic modeling process of a memristor includes the following specific steps:

[0132] Step S1: Repeat writing and measuring K times for any resistance state, and record the number of times each resistance state resistance value is accurately written; in this embodiment, the number of times each resistance state resistance value is accurately written is expressed as Acc = {k1, k2, ..., k N}, the writing accuracy of the resistance state will be represented by Acc;

[0133] Step S2: Based on the above read and write data, the statistical write error is modeled as a Gaussian noise function P(x);

[0134] Step S3: Count the damage of memristors in the memristor array, and based on the bad point distribution, model the probability of a bad point in the memristor as a salt and pepper noise function B(x);

[0135] In this embodiment, the target detection network weight update process includes the following specific steps:

[0136] Step S1', before the forward propagation of the target detection network training phase begins, all weights Weights(x) of the target detection network are retrieved;

[0137] Step S2', add the non-ideal characteristic noise P(x) and bad pixel noise B(x) of the memristor to the weights Weights(x), New (x) = Weights(x) + P(x) + B(x);

[0138] Step S3', set the new weight value WeightsNew (x) writing into each neuron of the target detection network;

[0139] Step S4', input training data, in the new weights Weights New Complete the forward propagation, calculate the gradient information based on the loss function, and update the weight value Weights New ;

[0140] Step S5': repeat steps S1' to S4'.

[0141] In this embodiment, the process of mapping the target detection network weight value to the on-chip memristor resistance value includes the following specific steps:

[0142] Step S1', take out the weights Weights_Final completed by the target detection network training, and calculate the maximum value weight_max and the minimum value weight_min of the weights;

[0143] Step S2”, calculate the maximum value R_max and the minimum value R_min of the resistance value that can be written to the memristor; in this embodiment, the maximum conductance value G that can be written to the memristor max =1 / R_min, the minimum conductivity is G min =1 / R_max;

[0144] Step S3″: Map the target detection network weight value weights_Final(i) to the conductance value G(i) of the memristor in the integrated sensing, storage and computing chip, and calculate it as follows:

[0145] G(i)=(G max -G min )·(weights_Final(i)-weight_min) / (weight_max-weitht-min)

[0146] Step S4”: convert all conductance values ​​G(i) into resistance values ​​R(i)=1 / G(i) of the memristor, and write the resistance values ​​R(i) into the corresponding memristors in the sensing-storage-computing integrated chip.

[0147] In summary, the present invention improves the system detection accuracy from two aspects: improving the weight characterization ability and overcoming the influence of the non-ideal characteristics of the memristor in the training process. It has better fault tolerance and higher detection accuracy. After the neural network weights are mapped to the sensor-storage-computing integrated chip, the disturbance of the memristor weights caused by the non-ideal characteristics of the memristor can be tolerated. The present invention has lower power consumption and faster calculation time. Compared with the implementation method based on NVIDIA graphics card on the computer, the processing time of the neural network structure in the hardware implementation method of the sensor-storage-computing integrated chip is the circuit response time, which can basically achieve instantaneous response. At the same time, the hardware simulation circuit does not require NVIDIA graphics card and CPU, and has lower power consumption.

[0148] The present invention has higher detection accuracy. During the weight training process, the present invention adopts a more complex network to guide the feature extraction process of the target detection network. Generally, within a certain range, the increase in the complexity of the network structure is conducive to improving its detection ability. Through the distillation method, a more complex network is used to guide the training process of the target detection network, which can improve the feature expression ability of the target detection network and thus improve the detection accuracy of the target detection network.

[0149] The present invention uses a memristor non-ideal characteristic modeling module to construct a mathematical model for the non-ideal characteristics of the memristor contained in the integrated sensing, storage and computing chip, such as resistance drift and resistance state instability; and integrates the non-ideal characteristics of the memristor into the training process of the target detection network, thereby overcoming the influence of the non-ideal characteristics of the memristor on the network detection accuracy.

[0150] The present invention quantifies the non-ideal characteristics of the memristor into a noise model and randomly adds the noise to the network weights during the training process, thereby integrating the non-ideal characteristics of the memristor into the neural network during the training process. The network has acquired the ability to resist weight perturbations. After the neural network weights are mapped to the integrated sensing, storage and computing chip, the perturbations of the memristor weights caused by the non-ideal characteristics of the memristor can be tolerated, thus having better fault tolerance.

[0151] The present invention solves the technical problem in the prior art that the non-ideal characteristics of the memristor lead to memristor weight disturbance, poor weight representation ability, and low target detection and recognition accuracy.

[0152] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A high-precision sensing and computing integrated target detection and recognition system, characterized by: The system comprises: A sensing, storage, and computing integrated chip memristor neural network circuit implementation module, used to construct the target detection network using a memristor array and applicable peripheral circuits; The target detection network knowledge distillation module is used to use a complex teacher network to guide the feature extraction process of the target detection network during training. The target detection network knowledge distillation module is connected to the memristor neural network circuit of the integrated sensing, storage and computing chip to achieve module connection; The memristor non-ideal characteristic modeling module is used to obtain the resistance drift parameter and instability parameter of the memristor from the integrated sensing, storage and computing chip, and to model the memristor's available resistance state, Gaussian noise function and salt and pepper noise function to construct a memristor non-ideal characteristic model; The target detection network weight fault tolerance module is used to utilize the memristor non-ideal characteristic model to convert the memristor non-ideal characteristics into superimposed noise on the target detection network weights; The target detection network weight fault tolerance module includes: a detection network weight extraction module, a memristor noise addition module, a weight value writing module, a weight update module and a loop update module; The detection network weight extraction module is used to obtain all initial network weights Weights(x) of the target detection network; The memristor noise adding module is used to add the Gaussian noise function P(x) and the salt and pepper noise function B(x) to the initial network weight Weights(x) to obtain a new weight value Weights New (x), the memristor noise adding module is connected to the detection network weight taking module: Weights New (x)=Weights(x)+P(x)+B(x); The weight value is written into the module to write the new weight value Weights New (x) writing into each neuron of the target detection network, wherein the weight value writing module is connected to the memristor noise adding module; The weight update module is used to input training data and update the weights according to the new weight values. New Perform the forward propagation operation of the target detection network, calculate the gradient information based on the preset loss function, and update the new weight value Weights accordingly New ; The cyclic update module repeatedly updates the new weight values ​​Weights New , based on which the applicable weights Weights_Final are obtained, the loop update module is connected to the weight update module; A target detection network weight value to on-chip memristor resistance mapping module is used to generate new weight values ​​using the superimposed noise, convert the new weight values ​​into memristor resistance values ​​in the sensing-storage-computing integrated chip, and construct a sensing-computing integrated target detection network based on them. The target detection network weight value to on-chip memristor resistance mapping module is connected to the target detection network weight fault tolerance module; The target detection network module is used to detect and identify the target to be detected from the input signal using the sensing and computing integrated target detection network. The target detection network module is connected to the target detection network weight value to the on-chip memristor resistance mapping module and the sensing, storage and computing integrated chip memristor neural network circuit module.

2. The high-precision sensing and computing integrated target detection and recognition system according to claim 1 is characterized in that: The target detection network includes: a feature encoding module, a feature decoding module, a category recognition module and a position prediction module; The feature encoding module uses the backbone network of ResNet18 to encode the light intensity information irradiated on the photoelectric memristor array to obtain light intensity encoded output information; The feature decoding module is used to decode and process the light intensity coded output information to obtain a light intensity decoding feature map, and the feature decoding module is connected to the feature coding module; The category recognition module is used to decode the feature map according to the light intensity, obtain and output a preset dimension vector, wherein the dimension of the preset dimension vector is adapted to the number of target categories to be detected; the category recognition module is connected to the feature decoding module; The position prediction module is used to perform position prediction based on the light intensity decoding feature map to obtain and output a Gaussian heat map, with the maximum value point of the Gaussian heat map as the target center position. The position prediction module is connected to the feature decoding module.

3. The high-precision sensing and computing integrated target detection and recognition system according to claim 1 is characterized in that: The sensing, storage and computing integrated chip memristor neural network circuit implementation module includes: a photoelectric memristor hardware perception module, a current signal conversion voltage signal module, an electrical state memristor feature extraction module and a recognition result output module; The photoelectric memristor hardware sensing module is used to sense and output a current signal I according to the input light signal. i ; including a photoelectric sensor array to correspond to the first convolutional layer of the target detection network; The current signal to voltage signal module is used to convert the current signal I i , performing a gain operation through a T-type structure and a preset resistor, thereby acquiring and outputting a voltage signal, and the current signal-to-voltage signal conversion module is connected to the photoelectric memristor hardware sensing module; The electrical state memristor feature extraction module is used to process the current signal I based on Kirchhoff's law using a preset extraction logic. i , based on which the electrical state memristor features are extracted, and the target detection network is used to perform weighted multiplication and accumulation operations: f(x,W)=W·x+b Where W represents the weight value of the target detection network, x is the network input, b is the bias, and the memristor voltage is multiplied by the conductance value to perform a convolution operation and cumulative summation operation. The current of the transformed parallel circuit is added to obtain the output current signal I out , as the electrical state memristor characteristic: I out =I1+I2+…+I n =U1·G1+U2·G2+…+U n ·G n Among them, U1,…,U n is the voltage value corresponding to the input x, G1,…,G n is the memristor conductance value corresponding to the weight W, I1,…,I n is the output result of the convolution operation; The recognition result output module is used to perform AD sampling on the electric state memristor feature to obtain and output a target recognition result. The recognition result output module is connected to the electric state memristor feature extraction module.

4. The high-precision sensing and computing integrated target detection and recognition system according to claim 3 is characterized in that: The current signal to voltage signal module includes: an operational amplifier, a first resistor, a second resistor, and a third resistor. The current signal to voltage signal module uses the following logic processing to obtain the voltage signal: V0=-I i (1+R2 / R3) R1 The first resistor R1, the second resistor R2 and the third resistor R3 form a T-shaped structure. The first end of the third resistor R3 is connected to the first resistor R1 and the second resistor R2. The second end of the third resistor R3 is grounded. The current signal I i They are respectively connected to the negative terminal of the operational amplifier and the first resistor R1, the positive terminal of the operational amplifier is grounded, and the second resistor R2 is connected to the output terminal of the operational amplifier.

5. The high-precision sensing and computing integrated target detection and recognition system according to claim 1 is characterized in that: The target detection network knowledge distillation module includes: the complex teacher network and the distillation loss function design module; The complex teacher network adopts the backbone network of ResNet101, and the complex teacher network includes: no less than 2 complex network convolution modules ConvBlock and no less than 2 downsampling layers MaxPooling; the complex network convolution module ConvBlock includes: Conv2D convolution and complex network activation layer Relu; The distillation loss function design module is used to obtain and calculate the knowledge distillation loss based on the KL divergence: Where p T is the probability predicted by the complex teacher network, p S is the probability predicted by the target detection network, represents the probability predicted by the complex teacher network at pixel t, Represents the probability predicted by the target detection network at pixel t, C is the predicted category, T represents the teacher network, and S represents the target detection network designed in this project.

6. The high-precision sensing and computing integrated target detection and recognition system according to claim 1 is characterized in that: The memristor non-ideal characteristic modeling module includes: a resistance value reading and writing data module, a write error modeling module and a memristor bad point modeling module; The resistance state value reading and writing data module is used to represent the available resistance states of the memristor using the following logic: R={R1,R2,…,R N } Where N is the number of available resistance states. Based on the available resistance states of the memristor, the writing operation into any resistance state is repeated and a preset number of measurements are performed to record the accurate number of times the resistance value of each resistance state is written. The following logic is then used to obtain the resistance state writing accuracy parameter Acc: Acc={k1,k2,…,k N } The write error modeling module is used to model the write accuracy parameter as the Gaussian noise function P(x), and the write error modeling module is connected to the resistive state resistance value reading and writing data module; The memristor bad point modeling module is used to count the damage of the memristors in the memristor array to obtain the distribution of memristor bad points, so as to obtain and model the probability of memristor bad points as the salt and pepper noise function B(x).

7. The high-precision sensing and computing integrated target detection and recognition system according to claim 1 is characterized in that: The target detection network weight value to the chip memristor resistance mapping module includes: a weight value processing module, a memristor resistance writing module, a detection network weight value mapping module and a memristor corresponding resistance writing module; The weight maximum value processing module is used to extract the applicable weights Weights_Final and calculate the maximum weight value Weight_max and the minimum weight value Weight_min based on them; The memristor resistance writing module is used to calculate the maximum value R_max and the minimum value R_min that can be written to the memristor resistance, and process them to obtain the maximum conductance value G of the memristor. max =1 / R_min and the minimum conductance of the memristor G min =1 / R_max; The detection network weight value mapping module is used to use the following logic to calculate the maximum conductance value G of the memristor according to the maximum conductance value G of the memristor. max =1 / R_min and the minimum conductance value of the memristor G min =1 / R_max, mapping the applicable network weight weights_Final(i) to the memristor conductance value G(i), and connecting the detection network weight value mapping module to the memristor resistance value writing module and the weight maximum value processing module; The memristor corresponding resistance writing module is used to convert all the memristor conductance values ​​G(i) into memristor resistance values ​​R(i), and write the memristor resistance values ​​R(i) into the corresponding memristors in the sensing, storage and computing integrated chip.

8. The high-precision sensing and computing integrated target detection and recognition system according to claim 7 is characterized in that: The detection network weight value mapping module maps the applicable network weight weights_Final(i) to the memristor conductance value G(i) using the following logic: G(i)=(G max -G min )·(weights_Fianl(i)-weight_min) / (weight_max-weight_min)。 9. A method for high-precision detection and identification of integrated sensing and computing targets, used to control the high-precision detection and identification system for integrated sensing and computing targets according to any one of claims 1 to 8, characterized in that: The method comprises: S1, constructing the target detection network using a memristor array and applicable peripheral circuits; S2. During the training process, a complex teacher network is used to guide the feature extraction process of the target detection network; S3. Obtaining resistance drift parameters and instability parameters of the memristor from the sensing-storage-computing integrated chip, and using these parameters to model the memristor's available resistance states, Gaussian noise function, and salt-and-pepper noise function, thereby constructing a non-ideal characteristic model of the memristor; S4, using the memristor non-ideal characteristic model, converting the memristor non-ideal characteristics into superimposed noise on the target detection network weights; S5. Generate a new weight value using the superimposed noise, convert the new weight value into a memristor resistance value in the sensing-storage-computing integrated chip, and construct a sensing-computing integrated target detection network based on the memristor resistance value. S6. Utilize the sensing-computing integrated target detection network to detect and identify the target to be detected from the input signal.

Images