Integrated sensing, storage and computing target detection and recognition system and method

Abstract

The present invention provides an integrated sensing, storage and computing target detection and recognition system and method. The system includes: a lightweight target detection network structure module for designing a lightweight target detection network. The lightweight target detection network structure module obtains non-redundant features and redundant features through convolution operations, and outputs the location category information of the target to be detected; a weight training module for obtaining the resistance instability parameters of the memristor to model a Gaussian noise model and train the network neuron weights of the lightweight target detection network; a weight deployment module for mapping and deploying the network neuron weights to the memristor array to obtain a suitable lightweight target detection network, so as to obtain the applicable location category information of the target to be detected and identify the integrated sensing, storage and computing target. The present invention solves the technical problems that the network weight disturbance leads to low recognition accuracy, and the high number of memristors required leads to high power consumption, large size and low degree of integration of the equipment.

Description

Technical Field

[0001] The present invention relates to the field of intelligent perception and image recognition technology, and specifically to an integrated sensing, storage and computing target detection and recognition system and method. Background Art

[0002] With the development of deep learning technology, network structures have become increasingly complex, and the number of parameters and floating-point operations required by neural networks has continued to increase. For example, the commonly used ResNet18 has 11.7M parameters, ResNet50 has 25M parameters, and ResNet101 has 44M parameters. The network multiplication and addition computational complexity is even greater. The latest lightweight networks, such as MobileNet V3, have an MAdds of 747.9M in the human key point detection task, and MicroNet has an MAdds of 77.7M. However, due to processing limitations, most current memristor arrays are less than 10M in size. For example, the memristor array published by Wu Huaqiang's team at Tsinghua University in Nature in 2020 contained 2048 neurons, the memristor array published by Miao Feng's team at Nanjing University in Nature Nanotechnology in 2021 was 32×32 in size, the memristor array published by Samsung Electronics in Nature in 2022 was 64×64 in size, and the memristor array published by the Korea Institute of Chemical Technology in Nature Communications was 320×320 in size.

[0003] The existing invention patent application document "Image feature selection method and module, neural network model based on memristor" with publication number CN114358146A, the method includes: obtaining a data set containing multiple images, each image containing M features, and the characteristic value of each feature is 0 or 1; counting the number of images Ni in the data set whose characteristic value of the i-th feature is 1, and setting an image number threshold Nth; constructing a memristor array containing 1 row × M columns, each memristor has the same pulse number threshold Pth, the initial state of the memristor is volatile and changes to non-volatile after receiving a pulse greater than or equal to the pulse number threshold Pth; applying Pi pulses to the memristor Ri in the i-th column to modulate the conductivity state of the corresponding memristor Ri, where Pi = (Pth / Nth) × Ni. And the existing invention patent application document "Operation method and data processing device of memristor array" with publication number CN115862708A, the method includes: the operation method of the memristor array includes a matrix-vector multiplication operation, and the matrix-vector multiplication operation includes: applying input voltages corresponding to input data of each row in the memristor array to word lines of each row in the memristor array; applying source line voltages to source lines of each column in the memristor array, so that the transistors in each memristor unit in each row of the memristor array are turned on and operate in the saturation region; applying a first fixed voltage to the bit lines of each row in the memristor array; and detecting the output current on the source lines of each column in the memristor array to obtain an output result.

[0004] The size of the memristor array within the aforementioned integrated sensing, storage, and computing chips in the prior art is still far smaller than the number of neurons required by a neural network. The current size of memristor arrays prevents most mainstream neural network frameworks from being directly deployed on them. Therefore, addressing this issue requires a more lightweight convolutional unit to reduce the number of neurons. At the same time, to maintain the original detection accuracy, this requires that when a neural network trained offline on a computer is deployed within the integrated sensing, storage, and computing chip, it can redundancy eliminate the network weight perturbations caused by the instability of the memristor resistance state, thereby maintaining the network's original detection accuracy.

[0005] In summary, the existing technology has technical problems such as low recognition accuracy caused by network weight disturbance, high device power consumption, large size and low degree of integration due to the high number of memristors required. Summary of the Invention

[0006] The technical problem to be solved by the present invention is: how to solve the technical problems in the prior art, such as low recognition accuracy caused by network weight disturbance, high power consumption, large size and low degree of integration of the device due to the high number of memristors required.

[0007] The present invention solves the above technical problems by adopting the following technical solutions: an integrated sensing, storage and computing target detection and recognition system includes:

[0008] The lightweight target detection network structure module includes: a feature encoding unit, a feature decoding unit and a target prediction unit; the feature encoding unit includes: a lightweight convolution unit and no less than 2 downsampling pooling functions; the feature decoding unit includes: a lightweight convolution unit and no less than 2 upsampling functions, and the feature decoding unit is connected to the feature encoding unit; the target prediction unit includes: a target position prediction module and a target category prediction module, and the target prediction unit is connected to the feature decoding unit. The lightweight target detection network structure module is used to design the convolution kernel size and the feature map generation method, and design the convolution method in combination with the characteristic parameters of the memristor to obtain a lightweight target detection network. The lightweight target detection network structure module detects the target location coordinates and category probability from the input image signal, and obtains non-redundant features and redundant feature cascades through convolution operations, and outputs the target location category information to be detected;

[0009] A weight training module is used to obtain the resistance instability parameters of the memristor to model a Gaussian noise model, which is used to train the network neuron weights of the lightweight target detection network. The weight training module is connected to the lightweight target detection network structure module.

[0010] The weight deployment module is used to map and deploy the network neuron weights to the memristor array of the integrated sensing, storage and computing chip, thereby obtaining a suitable lightweight target detection network. The suitable lightweight convolutional network is used to obtain the applicable location category information of the target to be detected, so as to identify and obtain the integrated sensing, storage and computing target. The weight deployment module is connected to the weight training module and the lightweight target detection network structure module.

[0011] The present invention starts from the solution idea of ​​lightweight network design, focuses on lightweight convolution unit design, lightweight network structure design, network weight training and deployment, and solves the problem of network structure inadaptability and reduced network detection accuracy caused by the limited number of memristors in the integrated sensing, storage and computing chip from an algorithmic perspective. The present invention uses lightweight convolution unit modules to design the convolution kernel size, the method of generating feature maps, and the convolution method combined with the memristor characteristic design to solve the problem that the scale of memristors in the memristor array in the existing integrated sensing, storage and computing chip cannot meet the number of neurons required by the conventional target detection network. The present invention improves the fault tolerance of the lightweight network to overcome the weight disturbance caused by the instability of the memristor resistance state by modeling the instability of the memristor resistance state as a Gaussian noise model, thereby improving the target recognition accuracy and making the system more compact.

[0012] In a more specific technical solution, the integrated sensing, storage and computing target detection and recognition system also includes: a photoelectric memristor sensing and computing integrated perception module circuit, a storage and computing integrated resistive memristor computing module circuit, and a prediction result acquisition module circuit:

[0013] The photoelectric memristor sensing and computing integrated perception module circuit includes: a light-sensitive array with adjustable light response;

[0014] A storage-computing integrated resistive-memristor computing module circuit includes a lightweight target detection network, the lightweight target detection network includes a resistive-memristor array for calculating the weights of network neurons other than the first layer of the lightweight target detection network, and a photoelectric-memristor sensing-computing integrated perception module circuit connected to the storage-computing integrated resistive-memristor computing module circuit;

[0015] The prediction result acquisition module circuit includes: a peripheral operational amplifier and a filter circuit. The prediction result acquisition module circuit is connected to the storage and calculation integrated resistive memristor calculation module circuit.

[0016] This invention features low power consumption and high-speed computing during target detection and recognition. By deploying neural network weights within an integrated sensing, storage, and computing chip, the convolution process's multiplication and accumulation operations are converted into a hardware circuit implementation based on Kirchhoff's principle. The convolution process is completed almost instantaneously, resulting in lower power consumption and faster computing speeds than traditional von Neumann-based neural network calculations on traditional computers.

[0017] In a more specific technical solution, the lightweight convolution unit includes: no less than 2 micro-convolutions, a convolution kernel size design component, a convolution kernel decomposition component, a jump connection setting component, and a redundant feature generation method component.

[0018] Convolution kernel size design component, used to design convolution kernels of preset sizes for convolutional neural networks;

[0019] The convolution kernel decomposition component is used to decompose the preset size convolution kernel to obtain at least two orthogonal convolutions, based on which sequential convolution is performed to obtain neighborhood information. The convolution kernel decomposition component is connected to the convolution kernel size design component;

[0020] The skip connection setting component is used to reduce the number of channels of sequential convolution, increase the width of lightweight convolution units and the number of sub-networks. The skip connection setting component is connected to the convolution kernel decomposition component;

[0021] The redundant feature generation method component is used to generate non-redundant features by orthogonal convolution in the lightweight convolution unit, and to generate redundant features Intrinsic_Features by micro-convolution to obtain and output non-redundant features Redundant_Features and redundant feature cascade Features:

[0022] Features=Concat(Intrinsic_Features, Redundant_Features).

[0023] The present invention solves the problem from two aspects: lightweight design of neural networks and improving the expression ability of lightweight networks. Through operations such as convolution kernel size design, convolution kernel orthogonal decomposition, skip connection setting and redundant feature generation method, the number of parameters required by the network is reduced exponentially compared with traditional convolution, and the deployment process requires fewer memristors, which solves the problem that the limited number of memristors in the current integrated sensing, storage and computing chip cannot meet the number of neurons required by most current mainstream neural networks, and can better adapt to the situation where the number of memristors in the current sensing, storage and computing chips is limited.

[0024] For memristor arrays, the present invention provides skip connections by simply adding connecting wires between rows and columns of memristors, thereby enhancing the characterization capability of the memristor network without increasing the number of memristors. Therefore, in the lightweight convolution unit module, the number of channels is reduced while increasing the skip connections between convolutions.

[0025] In a more specific technical solution, the lightweight convolution unit also includes: no less than 2 micro-convolution MConvBlocks, each micro-convolution MConvBlock includes but is not limited to: Conv2D with a convolution kernel size of 1×1 and an activation layer ReLu.

[0026] In a more specific technical solution, the weight training module includes: a memristor non-ideal characteristic noise generation module, a lightweight target detection network weight value and noise fusion module, and a weight update module;

[0027] A memristor non-ideal characteristic noise generation module, used to convert the non-ideal characteristic parameters of the memristor into Gaussian noise;

[0028] Lightweight target detection network weight value and noise fusion module, used to simulate the resistance instability of network weight mapping to memristor resistance, and process the network weight and Gaussian noise P G (z), based on which the new weight values ​​of the lightweight target detection network are obtained and output. new , the lightweight target detection network weight value and noise fusion module is connected to the memristor non-ideal characteristic noise generation module;

[0029] The weight update module is used to obtain the prediction error of the training data, calculate the gradient information based on it, and use the gradient information to update the new weight value Weights new , the weight update module is connected to the lightweight target detection network weight value and noise fusion module.

[0030] While the lightweight design of the present invention reduces the demand for the number of memristors, it still has high detection accuracy. In order to address the instability of the memristor configuration and ensure that the original detection accuracy of the network is maintained when the neural network trained offline on a computer is deployed in the integrated sensing, storage and computing chip, this system models the instability of the memristor resistance state as a noise model and integrates the weight generation process into the offline training process. This allows the lightweight network to be deployed in the integrated sensing, storage and computing chip, and can redundancy The network weight disturbance caused by the instability of the memristor resistance state. The present invention solves the problem of instability of the memristor configuration.

[0031] In a more specific technical solution, the memristor non-ideal characteristic noise generation module uses the following logic to convert the non-ideal characteristic parameters of the memristor into Gaussian noise P G (z):

[0032]

[0033] Wherein, σ represents the standard deviation, μ represents the mean, and x represents the weight of the network.

[0034] In a more specific technical solution, the lightweight target detection network weight value and noise fusion module uses the following logic to simulate the resistance instability of the network weight mapping to the memristor resistance value, and process the network weight and Gaussian noise P G (z), Weights represents the weight of the original lightweight target detection network, based on which the new weight value Weights of the lightweight target detection network is obtained and output new :

[0035] Weights new =Weights+P G (z).

[0036] In a more specific technical solution, the weight deployment module includes: a weight mapping module, a resistance value writing module, and a predicted current-to-voltage digitization module:

[0037] A weight mapping module, used to map the network neuron weights to the conductance values ​​of the memristors within the integrated sensing, storage, and computing chip;

[0038] The resistance writing module is used to write the resistance of the memristor to R i Write to the memristor at the corresponding position in the integrated sensing, storage and computing chip, and connect the resistance writing module to the weight mapping module;

[0039] Predict the current-to-voltage digitization module to calculate the current according to the memristor resistance R i , convert the predicted current output by the memristor crossbar array into a voltage value, and return the predicted result through AD sampling. The predicted current-to-voltage digitization module is connected to the resistance value writing module:

[0040]

[0041] Where, I1, I2, …, I n is the predicted current obtained by the multiplication and accumulation operation of the neural network, V in1 ,…,V imm is the signal voltage value loaded on the input end of the memristor array, g 11 ,…g mn Mapping neural network weights to the conductance values ​​of the memristor array.

[0042] In a more specific technical solution, the weight mapping module uses the following logic to map the network neuron weight weight_i to the memristor conductance value g:

[0043]

[0044] Where diff_g is the maximum modulatable range of the memristor conductance, weight_max is the maximum value of the neural network weight, and the memristor resistance R corresponding to the network neuron weight weight_i is i for:

[0045]

[0046] In a more specific technical solution, the integrated sensing, storage and computing target detection and recognition method includes:

[0047] S1. Design the convolution kernel size and feature map generation method, and design the convolution method based on the characteristic parameters of the memristor to obtain a lightweight target detection network. The lightweight target detection network structure module detects the target location coordinates and category probability from the input image signal, obtains non-redundant features and redundant features through convolution operations, and outputs the location category information of the target to be detected;

[0048] S2. Obtain the resistance instability parameters of the memristor to model a Gaussian noise model, which is used to train the network neuron weights of the lightweight target detection network;

[0049] S3. Map the network neuron weights to the memristor array of the integrated sensing, storage and computing chip to obtain a suitable lightweight target detection network. Use the suitable lightweight convolutional network to obtain the applicable location category information of the target to be detected, so as to identify and obtain the integrated sensing, storage and computing target.

[0050] Compared with the existing technology, the present invention has the following advantages: starting from the solution idea of ​​lightweight network design, the present invention focuses on lightweight convolution unit design, lightweight network structure design, network weight training and deployment, and solves the problem of network structure inadaptability and reduced network detection accuracy caused by the limited number of memristors in the integrated sensing, storage and computing chip from an algorithmic perspective. The present invention uses lightweight convolution unit modules to design the convolution kernel size, the method of generating feature maps, and the convolution method combined with the memristor characteristic design to solve the problem that the scale of memristors in the memristor array in the existing integrated sensing, storage and computing chip cannot meet the number of neurons required by the conventional target detection network. The present invention improves the fault tolerance of the lightweight network to overcome the weight disturbance caused by the instability of the memristor resistance state by modeling the instability of the memristor resistance state as a Gaussian noise model, thereby improving the target recognition accuracy and making the system more compact.

[0051] This invention features low power consumption and high-speed computing during target detection and recognition. By deploying neural network weights within an integrated sensing, storage, and computing chip, the convolution process's multiplication and accumulation operations are converted into a hardware circuit implementation based on Kirchhoff's principle. The convolution process is completed almost instantaneously, resulting in lower power consumption and faster computing speeds than traditional von Neumann-based neural network calculations on traditional computers.

[0052] The present invention solves the problem from two aspects: lightweight design of neural networks and improving the expression ability of lightweight networks. Through operations such as convolution kernel size design, convolution kernel orthogonal decomposition, skip connection setting and redundant feature generation method, the number of parameters required by the network is reduced exponentially compared with traditional convolution, and the deployment process requires fewer memristors, which solves the problem that the limited number of memristors in the current integrated sensing, storage and computing chip cannot meet the number of neurons required by most current mainstream neural networks, and can better adapt to the situation where the number of memristors in the current sensing, storage and computing chips is limited.

[0053] For memristor arrays, the present invention provides skip connections by simply adding connecting wires between rows and columns of memristors, thereby enhancing the characterization capability of the memristor network without increasing the number of memristors. Therefore, in the lightweight convolution unit module, the number of channels is reduced while increasing the skip connections between convolutions.

[0054] While the lightweight design of the present invention reduces the demand for the number of memristors, it still has high detection accuracy. In order to address the instability of the memristor configuration and ensure that the original detection accuracy of the network is maintained when the neural network trained offline on a computer is deployed in the integrated sensing, storage and computing chip, this system models the instability of the memristor resistance state as a noise model and integrates the weight generation process into the offline training process. This allows the lightweight network to be deployed in the integrated sensing, storage and computing chip, and can redundancy The network weight disturbance caused by the instability of the memristor resistance state. The present invention solves the problem of instability of the memristor configuration.

[0055] The present invention solves the technical problems existing in the prior art, such as low recognition accuracy caused by network weight disturbance, high power consumption, large size and low degree of integration of the device due to high requirement on the number of memristors. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a schematic diagram of data flow processing of the integrated sensing, storage and computing target detection and recognition system according to Example 1 of the present invention;

[0057] Figure 2 This is a schematic diagram showing the connection of basic modules of the integrated sensing, storage and computing target detection and recognition system according to Example 1 of the present invention;

[0058] Figure 3 Schematic diagram of the structure of a lightweight convolution unit according to Example 1 of the present invention;

[0059] Figure 4 Schematic diagram of the composition of convolutional orthogonal decomposition according to Example 1 of the present invention;

[0060] Figure 5 This is a schematic diagram of data flow processing of the weight deployment module of Example 1 of the present invention;

[0061] Figure 6 This is a schematic diagram of the basic steps of the integrated sensing, storage and computing target detection and recognition method of Example 2 of the present invention. DETAILED DESCRIPTION

[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0063] Example 1

[0064] like Figure 1 and Figure 2As shown, the integrated sensing, storage and computing target detection and recognition system provided by the present invention includes: a lightweight target detection network structure module 1, a weight training module 2 and a weight deployment module 3.

[0065] In this embodiment, the lightweight target detection network structure module 1 includes: a feature encoding unit 11 , a feature decoding unit 12 and a target prediction unit 13 .

[0066] In this embodiment, the feature encoding unit 11 includes but is not limited to: a lightweight convolution unit (LightConvBlock) and four downsampling pooling functions (MaxPooling);

[0067] In this embodiment, the feature decoding unit 12 includes but is not limited to: a lightweight convolution unit (LightConvBlock) and four upsampling functions (UpSampling), and the feature decoding unit 12 is connected to the feature encoding unit 11;

[0068] In this embodiment, the target prediction unit 13 includes but is not limited to: a target position prediction module 131 and a target category prediction module 132 . The target prediction unit 13 is connected to the feature decoding unit 12 .

[0069] In this embodiment, a lightweight convolution unit is used to design the convolution kernel size, the feature map generation method, and the convolution method combined with the characteristics of the memristor. The lightweight object detection network structure module 1 detects the location of the target from the input image signal and outputs the location and category information of the target to be detected.

[0070] like Figure 3 As shown, in this embodiment, the lightweight convolution unit (LightConvBlock) includes but is not limited to: convolution kernel size design, convolution orthogonal decomposition, skip connection (skip connection) and redundant feature generation method components. In this embodiment, the convolution kernel size design component commonly used in convolutional neural networks may include: a convolution kernel of size not less than 3×3, and the convolution kernel size is adapted to the number of memristors, so the lightweight convolution unit includes but is not limited to: M micro-convolution MConvBlocks. In this embodiment, each micro-convolution MConvBlock includes but is not limited to: a Conv2D with a convolution kernel size of 1×1 and an activation layer ReLu;

[0071] like Figure 4As shown, in this embodiment, the convolution kernel decomposition operation is performed using the convolution kernel decomposition component, and the convolution kernel of size k×k in the lightweight convolution unit is decomposed into two one-dimensional convolutions ColConvBlock and RowConvBlock in orthogonal directions. The one-dimensional convolutions ColConvBlock and RowConvBlock include: one one-dimensional convolution in an orthogonal direction (k×1, 1×k) and an activation layer Relu. The neighborhood information is obtained through the sequential convolution of the one-dimensional convolutions ColConvBlock and RowConvBlock.

[0072] In this embodiment, the skip connection setting component is used to increase the number of channels in the convolution process and increase the width of the convolution unit. This helps to improve the feature characterization capability of the convolution unit and improve the detection accuracy within a certain range, but the increase in the number of channels means that the number of memristors in a single memristor array must also increase accordingly. The skip connection can be regarded as increasing the width of the convolution unit and increasing the number of sub-networks. For the memristor array, the skip connection only needs to add connecting wires between the rows and columns of memristors, which enhances the characterization capability of the memristor network without increasing the number of memristors. Therefore, in the lightweight convolution unit, the number of channels will be reduced while increasing the skip connections between convolutions.

[0073] In this embodiment, in the redundant feature generation method component, the lightweight convolution unit uses the aforementioned orthogonal convolution ColConvBlock and RowConvBlock to generate a portion of non-redundant features Intrinsic_Features, and the remaining redundant features Redundant_Features are obtained by the aforementioned micro-convolution MConvBlock with a convolution kernel size of 1×1. The feature map Features output by the lightweight convolution unit is a cascade of non-redundant features and redundant features:

[0074] Features=Concat(Intrinsic_Features, Redundant_Features).

[0075] In this embodiment, the output feature map F of the feature decoding unit Decoder It will serve as the input of both the target position prediction module 131 and the target category prediction module 132. In this embodiment, the output of the target position prediction module 131 is the xy coordinates of the location of the target to be detected, and the output of the target category prediction module 132 is the probability of the category to which the target to be detected belongs.

[0076] In this embodiment, the weight training module 2 is used to train the lightweight target detection network weights, and the instability of the memristor resistance state is modeled as a Gaussian noise model. In this embodiment, the weight training module 2 includes: a memristor non-ideal characteristic noise generation module 21, a lightweight target detection network weight value and noise fusion module 22, and a weight update module 23;

[0077] In this embodiment, the non-ideal characteristics of the memristor are modeled as a noise model using the memristor non-ideal characteristic noise generation module 21, and the non-ideal characteristics of the memristor are converted into Gaussian noise through the memristor non-ideal characteristic noise generation module 21:

[0078]

[0079] In this embodiment, the lightweight target detection network weights (Weights) and noise P G (z) Fusion module 22 simulates the resistance instability of the superimposed resistance state during the weight mapping to the resistance value of the memristor. The lightweight target detection network weight value and noise fusion module 22 uses the network weight value Weights and Gaussian noise P G (z) is used as input, and the output is the new weight value of the lightweight target detection network. The lightweight target detection network weight value is connected with the noise fusion module 22 and the memristor non-ideal characteristic noise generation module 21:

[0080] Weights new =Weights+P G (z).

[0081] In this embodiment, the weight update module 23 is used to update the weight values ​​​​of the training data prediction error based on the gradient information. new to update.

[0082] In this embodiment, weight deployment module 3 maps and deploys the neuron weights trained in the aforementioned lightweight object detection network onto the memristor array of the integrated sensing, storage, and computing chip. In this embodiment, each neuron in the lightweight neural network corresponds to a memristor device in the memristor array. In this embodiment, the neuron weights are mapped to the conductance values ​​of the corresponding memristor devices.

[0083] In this embodiment, the weight deployment module 3 includes but is not limited to: a weight mapping module 31, a resistance value writing module 32, and a predicted current-to-voltage digitization module 33:

[0084] like Figure 5As shown, in this embodiment, the weight value of the target detection network is mapped to the resistance value of the memristor in the integrated sensing, storage and computing chip using the weight mapping module 31. The maximum value of the weight of the target detection network is weight_max, and the minimum adjustable resistance value of the memristor is R min , maximum resistance R max , its conductance minimum Conductivity maximum Conductivity change range diff_g = g max -g min , the weights weight_i of the above target detection network are mapped to the conductance values The memristor resistance corresponding to the weight weight_i

[0085] In this embodiment, the resistance writing module 32 is used to write the memristor resistance R corresponding to the weight weight_i calculated above. i The resistance value is written to the memristor at the corresponding position in the integrated sensing, storage and computing chip, and the resistance value writing module 32 is connected to the weight mapping module 31;

[0086] In this embodiment, the predicted current to voltage digitalization module 33 is used to convert the predicted current ([I1, I2, ..., I n ]) is converted into a voltage value, and through AD sampling, the prediction result is returned. The predicted current-to-voltage digitization module 33 is connected to the resistance value writing module 32:

[0087]

[0088] In this embodiment, the sensing, storage and computing integrated target detection and recognition system also includes the following hardware components: a photoelectric memristor sensing and computing integrated perception module circuit, a storage and computing integrated resistive memristor computing module circuit, and a prediction result acquisition module circuit:

[0089] In this embodiment, the integrated photoelectric memristor sensing and computing module circuit includes a photoelectric memristor array with modulatable photoresponsivity, which serves as a photosensitive array, with each photoelectric memristor acting as a pixel in the photosensitive array. In this embodiment, the photoelectric memristor has at least 10 adjustable photoresponsivity states, and the photoresponse state values ​​are derived from the weight mapping of the first layer of the lightweight object detection network.

[0090] In this embodiment, the integrated memory and computing resistive memristor computing module circuit includes a lightweight target detection network, which includes a resistive memristor array. In this embodiment, the integrated memory and computing resistive memristor computing module circuit is used to calculate the network weights of the lightweight target detection network except for the first layer, and the weight mapping module is used to map the weight values ​​of the lightweight target detection network to the resistance values ​​of the memristors.

[0091] In this embodiment, the prediction result acquisition module circuit includes a prediction current-to-voltage digitization module, a peripheral operational amplifier, and a filter circuit.

[0092] Example 2

[0093] like Figure 6 As shown, in this embodiment, the integrated sensing, storage and computing target detection and recognition method provided by the present invention includes the following basic steps:

[0094] Step S1: Create a training data set, collect training images containing the target to be detected, and manually mark the location of the target;

[0095] Step S2: using the above training data to train the lightweight neural network;

[0096] Step S3: superimposing the noise constructed based on configuration instability onto the weights of the lightweight neural network, calculating the gradient information according to the prediction error, and updating the weight values ​​of the lightweight neural network;

[0097] Step S4: Repeat steps S2 and S3 until the loss value in the training process no longer decreases, and the training ends;

[0098] Step S5: Deploy the lightweight neural network weights that have completed the training into the integrated sensing, storage and computing chip.

[0099] In summary, the present invention starts from the solution idea of ​​lightweight network design, focuses on lightweight convolution unit design, lightweight network structure design, network weight training and deployment, and solves the problem of network structure inadaptability and reduced network detection accuracy caused by the limited number of memristors in the integrated sensing, storage and computing chip from an algorithmic perspective. The present invention uses lightweight convolution unit modules to design the convolution kernel size, the method of generating feature maps, and the convolution method combined with the memristor characteristic design to solve the problem that the scale of memristors in the memristor array in the existing integrated sensing, storage and computing chip cannot meet the number of neurons required by the conventional target detection network. The present invention improves the fault tolerance of the lightweight network to overcome the weight disturbance caused by the instability of the memristor resistance state by modeling the instability of the memristor resistance state as a Gaussian noise model, thereby improving the target recognition accuracy and making the system more compact.

[0100] This invention features low power consumption and high-speed computing during target detection and recognition. By deploying neural network weights within an integrated sensing, storage, and computing chip, the convolution process's multiplication and accumulation operations are converted into a hardware circuit implementation based on Kirchhoff's principle. The convolution process is completed almost instantaneously, resulting in lower power consumption and faster computing speeds than traditional von Neumann-based neural network calculations on traditional computers.

[0101] The present invention solves the problem from two aspects: lightweight design of neural networks and improving the expression ability of lightweight networks. Through operations such as convolution kernel size design, convolution kernel orthogonal decomposition, skip connection setting and redundant feature generation method, the number of parameters required by the network is reduced exponentially compared with traditional convolution, and the deployment process requires fewer memristors, which solves the problem that the limited number of memristors in the current integrated sensing, storage and computing chip cannot meet the number of neurons required by most current mainstream neural networks, and can better adapt to the situation where the number of memristors in the current sensing, storage and computing chips is limited.

[0102] For memristor arrays, the present invention provides skip connections by simply adding connecting wires between rows and columns of memristors, thereby enhancing the characterization capability of the memristor network without increasing the number of memristors. Therefore, in the lightweight convolution unit module, the number of channels is reduced while increasing the skip connections between convolutions.

[0103] While the lightweight design of the present invention reduces the demand for the number of memristors, it still has high detection accuracy. In order to address the instability of the memristor configuration and ensure that the original detection accuracy of the network is maintained when the neural network trained offline on a computer is deployed in the integrated sensing, storage and computing chip, this system models the instability of the memristor resistance state as a noise model and integrates the weight generation process into the offline training process. This allows the lightweight network to be deployed in the integrated sensing, storage and computing chip, and can redundancy The network weight disturbance caused by the instability of the memristor resistance state. The present invention solves the problem of instability of the memristor configuration.

[0104] The present invention solves the technical problems existing in the prior art, such as low recognition accuracy caused by network weight disturbance, high power consumption, large size and low degree of integration of the device due to high requirement on the number of memristors.

[0105] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An integrated sensing, storage and computing target detection and recognition system, characterized by: The system comprises: The lightweight target detection network structure module includes: a feature encoding unit, a feature decoding unit and a target prediction unit; the feature encoding unit includes: a lightweight convolution unit and no less than 2 downsampling pooling functions; the feature decoding unit includes: a lightweight convolution unit and no less than 2 upsampling functions, and the feature decoding unit is connected to the feature encoding unit; the target prediction unit includes: a target position prediction module and a target category prediction module, and the target prediction unit is connected to the feature decoding unit. The lightweight target detection network structure module is used to design the convolution kernel size and the feature map generation method, and design the convolution method in combination with the characteristic parameters of the memristor to obtain a lightweight target detection network. The lightweight target detection network structure module detects the target location coordinates and category probability from the input image signal, and obtains non-redundant features and redundant feature cascades through convolution operations, and outputs the target location category information to be detected; The lightweight convolution unit includes: no less than 2 micro-convolutions, a convolution kernel size design component, a convolution kernel decomposition component, a skip connection setting component, and a redundant feature generation method component; Convolution kernel size design component, used to design convolution kernels of preset sizes for convolutional neural networks; The convolution kernel decomposition component is used to decompose the preset size convolution kernel to obtain at least two orthogonal convolutions, based on which sequential convolution is performed to obtain neighborhood information. The convolution kernel decomposition component is connected to the convolution kernel size design component; The skip connection setting component is used to reduce the number of channels of sequential convolution, increase the width of lightweight convolution units and the number of sub-networks. The skip connection setting component is connected to the convolution kernel decomposition component; The redundant feature generation method component is used to generate non-redundant features by orthogonal convolution in the lightweight convolution unit, and to generate redundant features by micro-convolution, so as to obtain and output the cascade of non-redundant features and redundant features: Features=Concat(Intrinsic_Features,Redundant_Features); A weight training module is used to obtain the resistance instability parameters of the memristor to model a Gaussian noise model, which is used to train the network neuron weights of the lightweight target detection network. The weight training module is connected to the lightweight target detection network structure module. The weight training module includes: a memristor non-ideal characteristic noise generation module, a lightweight target detection network weight value and noise fusion module, and a weight update module; A memristor non-ideal characteristic noise generation module, used to convert the non-ideal characteristic parameters of the memristor into Gaussian noise; Lightweight target detection network weight value and noise fusion module, used to simulate the resistance instability of network weight mapping to memristor resistance, and process the network weight and Gaussian noise P G (z), based on which the new weight values ​​of the lightweight target detection network are obtained and output. new , the lightweight target detection network weight value and noise fusion module is connected to the memristor non-ideal characteristic noise generation module; The weight update module is used to obtain the prediction error of the training data, calculate the gradient information based on it, and use the gradient information to update the new weight value Weights new , the weight update module is connected with the lightweight target detection network weight value and noise fusion module; The weight deployment module is used to map and deploy the network neuron weights to the memristor array of the integrated sensing, storage and computing chip, thereby obtaining a suitable lightweight target detection network. The suitable lightweight convolutional network is used to obtain the applicable location category information of the target to be detected, so as to identify and obtain the integrated sensing, storage and computing target. The weight deployment module is connected to the weight training module and the lightweight target detection network structure module.

2. The integrated sensing, storage and computing target detection and recognition system according to claim 1 is characterized in that: The integrated sensing, storage and computing target detection and recognition system also includes: a photoelectric memristor sensing and computing integrated perception module circuit, a storage and computing integrated resistive memristor computing module circuit and a prediction result acquisition module circuit. The photoelectric memristor sensing and computing integrated perception module circuit includes: a light-sensitive array with adjustable light responsivity; The storage-computing integrated resistive-memristor computing module circuit includes: the lightweight target detection network, the lightweight target detection network includes: a resistive-memristor array for calculating the weights of the network neurons of the lightweight target detection network except the first layer, and the photoelectric-memristor sensing-computing integrated perception module circuit is connected to the storage-computing integrated resistive-memristor computing module circuit; The prediction result acquisition module circuit includes: a peripheral operational amplifier and a filter circuit, and the prediction result acquisition module circuit is connected to the storage and calculation integrated resistive-state memristor calculation module circuit.

3. The integrated sensing, storage and computing target detection and recognition system according to claim 1 is characterized in that: The lightweight convolution unit further includes: no less than 2 micro-convolution MConvBlocks, each of the micro-convolution MConvBlocks includes but is not limited to: Conv2D with a convolution kernel size of 1×1 and an activation layer ReLu.

4. The integrated sensing, storage and computing target detection and recognition system according to claim 1 is characterized in that: The memristor non-ideal characteristic noise generation module uses the following logic to convert the non-ideal characteristic parameters of the memristor into Gaussian noise P G (z):

5. The integrated sensing, storage and computing target detection and recognition system according to claim 1 is characterized in that: The lightweight target detection network weight value and noise fusion module uses the following logic to simulate the resistance instability of the network weight mapped to the memristor resistance value, and processes the network weight and Gaussian noise P ， (z), based on which the new weight values ​​of the lightweight target detection network are obtained and output. new : Weights new =Weights+P G (z) Where Weights represents the weights of the original lightweight target detection network.

6. The integrated sensing, storage and computing target detection and recognition system according to claim 1 is characterized in that: The weight deployment module includes: a weight mapping module, a resistance value writing module, and a predicted current-to-voltage digitization module: The weight mapping module is used to map the network neuron weights to the memristor conductance values ​​in the sensing, storage and computing integrated chip; The resistance writing module is used to write the resistance value R i Writing into the memristor at the corresponding position in the sensing-storage-computing integrated chip, the resistance writing module is connected to the weight mapping module; The predicted current-to-voltage digitization module is used to convert the current into voltage according to the resistance value R of the memristor. i , converting the predicted current output by the memristor crossbar array into a voltage value, and returning the predicted result through AD sampling, the predicted current-to-voltage digitization module is connected to the resistance value writing module: Where, I1, I2, …, I n is the predicted current obtained by the multiplication and accumulation operation of the neural network, V in1 ,…,V inm is the signal voltage value loaded on the input end of the memristor array, g 11 ,…g mn Mapping neural network weights to the conductance values ​​of the memristor array.

7. The integrated sensing, storage and computing target detection and recognition system according to claim 6 is characterized in that: The weight mapping module maps the network neuron weight weight_i to the memristor conductance value using the following logic: Where diff_g is the maximum modulatable range of the memristor conductance, weight_max is the maximum value of the neural network weight, and the memristor resistance corresponding to the network neuron weight weight_i is:

8. An integrated sensing, storage and computing target detection and recognition method for controlling the integrated sensing, storage and computing target detection and recognition system according to any one of claims 1 to 7, characterized in that: The method comprises: S1. Designing the convolution kernel size and the method for generating the feature map, and designing the convolution method based on the characteristic parameters of the memristor, thereby obtaining a lightweight target detection network. The lightweight target detection network structure module detects the target location coordinates and category probability from the input image signal, obtains non-redundant features and redundant features through convolution operations, and outputs the category information of the target location to be detected; S2. Obtaining a resistance-state instability parameter of the memristor to model a Gaussian noise model, and using the model to train the network neuron weights of the lightweight target detection network; S3. Map the network neuron weights and deploy them to the memristor array of the sensing, storage and computing integrated chip to obtain a suitable lightweight target detection network. Utilize the suitable lightweight convolutional network to obtain the applicable location category information of the target to be detected, so as to identify and obtain the integrated sensing, storage and computing target.

Images