Image recognition method and apparatus based on physical reservoir

Abstract

This disclosure provides an image recognition method and apparatus based on a physical reservoir. The method includes: constructing an image recognition model based on a physical reservoir; the image recognition model includes at least an input layer, a reservoir layer, and an output layer, wherein the input layer is used to convert the input image into an input signal; the reservoir layer includes N resistor-capacitor series circuits, each corresponding to a processing channel for nonlinear mapping of the input signal; the output layer is used to output the recognition result; the image recognition model is trained using labeled sample images; the image to be recognized is input into the image recognition model, and the output result of the image recognition model is determined as the image recognition result. This disclosure, through the nonlinear characteristics and short-time memory capability of capacitance, can effectively achieve dynamic data capture by the reservoir layer, reduce the number of weights required for training, and simultaneously ensure high accuracy of the image recognition result.

Description

Technical Field

[0001] This disclosure relates to the field of image recognition technology, and in particular to an image recognition method and apparatus based on a physical reservoir. Background Technology

[0002] Image recognition, a key area of ​​artificial intelligence, has achieved significant performance improvements through deep learning techniques such as Convolutional Neural Networks (CNNs), Transformers, and Deep Belief Networks (DBNs). However, these techniques are limited in their application on resource-constrained devices, primarily due to their high power consumption and computational cost. To address this, neuromorphic computing has emerged, simulating biological nervous systems to improve computational efficiency and reduce energy consumption. Reservoir Computing (RC), a representative of neuromorphic computing, consists of an input layer, a reservoir, and an output layer. Its low training cost and efficient processing capabilities for dynamic data have made it a research hotspot in the field of image recognition.

[0003] In the implementation of physical storage networks, memristors have become the mainstream choice for construction due to their unique memory characteristics and nonlinear mapping capabilities. Memristors demonstrate advantages in energy efficiency and speed in applications such as image recognition; however, their manufacturing process is complex and costly, and the nonlinear output of memristors can lead to different outputs when the same input data is used, affecting the predictability and reliability of the system. Therefore, existing physical storage systems still face challenges in terms of practical application and data processing accuracy. Summary of the Invention

[0004] The purpose of this disclosure is to provide an image recognition method and apparatus based on physical storage pools, so as to solve the problems existing in the prior art that use memristors as physical storage pool networks.

[0005] The embodiments of this disclosure adopt the following technical solution: an image recognition method based on a physical reservoir, comprising: constructing an image recognition model based on a physical reservoir; the image recognition model includes at least an input layer, a reservoir layer, and an output layer, wherein the input layer is used to convert the input image into an input signal; the reservoir layer includes N resistor-capacitor series circuits, each resistor-capacitor series circuit corresponding to a processing channel for nonlinear mapping of the input signal; the output layer is used to output the recognition result; training the image recognition model using labeled sample images; inputting the image to be recognized into the image recognition model, and determining the output result of the image recognition model as the recognition result of the image.

[0006] In some embodiments, the input layer is specifically used to perform any one or more of the following on the input image: image reshaping, masking, and binarization.

[0007] In some embodiments, the image to be identified includes at least simple type images and complex type images; wherein, the simple type images include at least 0 to 9 digit images composed of pixels, and the complex type images include at least handwritten digit images, handwritten letter images, and clothing images.

[0008] In some embodiments, the input signal is N PWM signals, and the pulse width of each PWM signal is proportional to the pixel value of the corresponding pixel in the input image.

[0009] In some embodiments, the reservoir layer performs nonlinear mapping on the preprocessed image, including: inputting N PWM signals into their corresponding resistor-capacitor series circuits for short-term memory, selecting the end time of a preset pulse period in each PWM signal as a virtual node, and determining the capacitor voltage value at all the virtual nodes.

[0010] In some embodiments, when the image to be identified is a simple type image, the preset pulse period is the last pulse period in the PWM signal; when the image to be identified is a complex type image, the preset pulse period is each pulse period in the PWM signal.

[0011] In some embodiments, the capacitor voltage value at the virtual node is determined based on the following formula:

[0012]

[0013] Among them, V c (t0) represents the capacitor voltage value read at the virtual node; V in The input voltage is represented by T; the duration of a single pulse cycle is represented by t0; the duration of the high level in a single pulse cycle is represented by τ; and the time parameter is determined by the hardware parameters of the resistor-capacitor circuit, τ = R * C, where R is the resistance and C is the capacitance.

[0014] In some embodiments, the output layer outputs the recognition result, including: performing matrix operations on the capacitor voltage values ​​at all the virtual nodes and the trained weight matrix, determining the label value of the result using a winner-takes-all strategy, and determining the image recognition result based on the label value.

[0015] In some embodiments, training the image recognition model using labeled sample images includes training the weight matrix using linear regression.

[0016] This disclosure also provides an image recognition device based on a physical reservoir, comprising: a model building module for building an image recognition model based on a physical reservoir; the image recognition model includes at least an input layer, a reservoir layer, and an output layer, wherein the input layer is used to preprocess the input image; the reservoir layer includes N resistor-capacitor series circuits, each resistor-capacitor series circuit corresponding to a processing channel for nonlinear mapping of the preprocessed image; the output layer is used to output the recognition result; a model training module for training the image recognition model using labeled sample images; and an image recognition module for inputting the image to be recognized into the image recognition model and determining the output result of the image recognition model as the recognition result of the image.

[0017] The beneficial effects of the embodiments disclosed herein are as follows: using a resistor-capacitor series circuit as the reservoir layer of the physical reservoir for image recognition makes the reservoir network structure simple and easy to build; in addition, through the nonlinear characteristics and short-time memory capability of the capacitor, the reservoir can effectively achieve dynamic capture of data, reduce the number of weights required for training, and at the same time ensure high accuracy of image recognition results. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of the image recognition method based on a physical storage tank in the first embodiment of this disclosure;

[0020] Figure 2 (a) to (d) are schematic diagrams of the simulation process of Example 1 in the first embodiment of this disclosure;

[0021] Figure 3 This is a comparison of simulation and actual measurement results for Example 1 in the first embodiment of this disclosure;

[0022] Figure 4 This is a schematic diagram of the simulation process of Example 2 in the first embodiment of this disclosure;

[0023] Figure 5 This is a schematic diagram illustrating the accuracy of handwritten digit recognition in the first embodiment of this disclosure;

[0024] Figure 6 This is a schematic diagram showing the comparison results of PAM and PWM in the first embodiment of this disclosure;

[0025] Figure 7 This is a schematic diagram of the handwritten letter recognition results in the first embodiment of this disclosure;

[0026] Figure 8 This is a schematic diagram of the clothing recognition results in the first embodiment of this disclosure;

[0027] Figure 9 This is a schematic diagram of the structure of the image recognition device based on a physical reservoir in the second embodiment of this disclosure. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0029] Image recognition, a key area of ​​artificial intelligence, has achieved significant performance improvements through deep learning techniques such as Convolutional Neural Networks (CNNs), Transformers, and Deep Belief Networks (DBNs). However, these techniques are limited in their application on resource-constrained devices, primarily due to their high power consumption and computational cost. To address this, neuromorphic computing has emerged, simulating biological nervous systems to improve computational efficiency and reduce energy consumption. Reservoir Computing (RC), a representative of neuromorphic computing, consists of an input layer, a reservoir, and an output layer. Its low training cost and efficient processing capabilities for dynamic data have made it a research hotspot in the field of image recognition.

[0030] In the implementation of physical storage networks, memristors have become the mainstream choice for construction due to their unique memory characteristics and nonlinear mapping capabilities. Memristors demonstrate advantages in energy efficiency and speed in applications such as image recognition; however, their manufacturing process is complex and costly, and the nonlinear output of memristors can lead to different outputs when the same input data is used, affecting the predictability and reliability of the system. Therefore, existing physical storage systems still face challenges in terms of practical application and data processing accuracy.

[0031] The first embodiment of this disclosure provides an image recognition method based on a physical storage pool. This method is primarily used to recognize two types of images: simple images, which include at least images of digits 0 to 9 composed of pixels, and may also include letter images composed of pixels, etc., whose content can be clearly defined using only black and white pixels; and complex images, which include at least handwritten digit images, handwritten letter images, clothing images, etc., whose content cannot be accurately described using only black and white pixels. The flowchart of this embodiment is shown below. Figure 1 As shown, the main steps include the following:

[0032] S10, Construct an image recognition model based on a physical reservoir; the image recognition model includes at least an input layer, a reservoir layer, and an output layer, wherein the input layer is used to convert the input image into an input signal; the reservoir layer includes N resistor-capacitor series circuits, each resistor-capacitor series circuit corresponding to a processing channel, used to perform nonlinear mapping on the preprocessed image; the output layer is used to output the recognition result; N is a positive integer;

[0033] S20, The image recognition model is trained using labeled sample images;

[0034] S30: Input the image to be recognized into the image recognition model, and determine the output of the image recognition model as the recognition result of the image.

[0035] The image recognition model constructed in this embodiment is a physical reservoir network model implemented based on a resistor-capacitor series circuit. It mainly includes an input layer, a reservoir layer, and an output layer. The input and output layers can be implemented using computer software, while the reservoir layer is implemented based on a physical resistor-capacitor series circuit mounted on a breadboard. It is important to note that the number of resistor-capacitor series circuits varies depending on the number of channels in the image recognition model. Each channel requires a resistor-capacitor series circuit for nonlinear mapping of the data within that channel. The output layer outputs the recognition result based on the unified output results from all channels.

[0036] The following section, in conjunction with an introduction to the functions of each level in the image recognition model, explains the implementation process of the image recognition method using a physical reservoir.

[0037] Specifically, considering the signal input format of the reservoir layer, the input signal obtained by the input layer should be provided to the reservoir layer in the form of a voltage signal. Corresponding to the image input into the model, the input layer should transform it into an image with a specific number of pixel rows and each pixel within a specific pixel value range through specific image processing methods, so that the reservoir layer can perform subsequent signal mapping and capture the correlation between data. In some embodiments, the input layer can process the input image through any one or more of shaping, masking, and binarization. Different preprocessing methods can also be used for different types of images to be recognized. For simple types of images, binarization is sufficient; while for complex types of images, shaping and masking are required, that is, transforming the image input to the model into a specific size (i.e., having a specific number of pixel rows and columns) and normalizing the pixel values ​​of all pixels in the image to a specific pixel value range.

[0038] Furthermore, to fully utilize the nonlinear characteristics of the resistor-capacitor series circuit, the input signal generated by the input layer in this embodiment can preferably be a PWM (Pulse Width Modulation) signal, and the pulse width in the PWM signal represents the pixel value of the corresponding pixel in the input image. Specifically, in this embodiment, different pixel rows are used as different calculation channels, so each row of pixels in the input image corresponds to a PWM signal, and the pulse width in each PWM signal is proportional to the pixel value of the pixel in its corresponding pixel row.

[0039] The process of nonlinear mapping of the preprocessed image by the reservoir layer mainly includes: inputting N PWM signals into their corresponding resistor-capacitor series circuits for short-term memory, and selecting the end time of the preset pulse period in each PWM signal as a virtual node, determining the capacitor voltage value at all virtual nodes. It is important to note that the introduction of virtual nodes is mainly used to increase the reservoir size, obtain more mapping data, and improve the model's output capability without additional physical reservoir settings. In actual implementation, virtual nodes can be selected in different ways for different types of input images. For example, when the image to be identified is a simple type, the preset pulse period is the last pulse period of the PWM signal, which is equivalent to taking the capacitor voltage after all pulses of the corresponding PWM signal have been input for each channel; when the image to be identified is a complex type, the preset pulse period is each pulse period of the PWM signal, which is equivalent to outputting a result for each pulse input of the PWM signal. Specifically, the capacitor voltage value at the virtual node is determined based on the following formula:

[0040]

[0041] Among them, V c (t0) represents the capacitor voltage value read at the virtual node; V in The input voltage is represented by t0, which does not affect the output. T represents the duration of a single pulse cycle, which is the sum of the charging and discharging times. t0 represents the duration of the high level in a single pulse cycle. τ represents the time parameter, which is determined by the hardware parameters of the resistor-capacitor circuit. The calculation method is τ = R * C, where R is the resistance and C is the capacitance.

[0042] After the reservoir layer outputs the capacitor voltage values ​​at each virtual node, the output layer, based on the output of the reservoir layer, performs matrix operations with the trained weight matrix, uses a winner-takes-all strategy to determine the label value of the result, and determines the image recognition result based on the label value. In this embodiment, the training of the image recognition model in step S20 mainly refers to training the weight matrix. The labeled sample images can come from the handwritten digit set (MNIST, Mixed National Institute of Standards and Technology database), the handwritten letter set (E-MNIST, an extension of MNIST to handwritten letters), the clothing dataset (Fashion-MNIST, a Novel Image Dataset for Benchmarking Machine Learning Algorithms), etc. If simple image recognition is required, clear digit or letter images can be directly used for model training.

[0043] The weight matrix is ​​trained using linear regression, such as the least squares method, or other training methods with the same function, such as ridge regression. Formula (2) shows the formula for calculating the weight matrix using linear regression:

[0044] W = Y × τ T ×(X×X T ) + (2)

[0045] Where X represents the matrix output by all sample images after passing through the resistor-capacitor series circuit, which is also the matrix of capacitor voltage values ​​output by the reservoir layer. The superscript T indicates the transpose of the matrix, the superscript + indicates the pseudo-inverse of the matrix, Y is the label value matrix after one-hot encoding, and W is the weight matrix.

[0046] This embodiment utilizes a resistor-capacitor series circuit as the reservoir layer of the physical reservoir for image recognition, which simplifies the reservoir network structure and makes it easy to build. Only a few resistor-capacitor series circuits need to be built, and the rest can be implemented by computer software, thus reducing costs. In addition, through the nonlinear characteristics and short-time memory capability of capacitors, the reservoir can effectively capture data dynamically, reduce the number of weights required for training, and ensure high accuracy of image recognition results.

[0047] The following is combined Figures 2 to 8 The image recognition method based on physical reservoirs in this embodiment is illustrated through two simulation examples.

[0048] Example 1

[0049] For simple image types, Figure 2 Taking the number 5 shown in (a) as an example, black pixels in the image are represented as low levels in the PWM signal, and white pixels are represented as high levels in the PWM signal. The image is then divided into five channels according to pixel rows, forming a pattern as shown in the diagram. Figure 2 (b) shows the pulse configuration of the five PWM signals. The duration of each pulse cycle is set to 0.5τ, and the duration of the high-level pulse is 0.2τ. The five PWM signals are input into five resistor-capacitor series circuits, as shown below. Figure 2 As shown in (c), the final voltage value of the entire pulse line is recorded at time 2τ, as follows. Figure 2 As shown in (d), this value is used as the output of the resistor-capacitor series circuit. The simple digits composed of 5 rows of pixels are finally converted into five voltage values. The 10 simple digits used for training (0 to 9 respectively) are converted into a (10, 5) matrix X after passing through the resistor-capacitor series circuit. After adding a bias, the weight matrix W is calculated using linear regression (formula (2)). Then, the trained weight matrix is ​​used to test the noisy test set. The obtained values ​​are used to obtain the final result through the winner-take-all method to verify the effectiveness of the model.

[0050] The resistor used in this example to form the resistor-capacitor series circuit is 4.7 x 10⁻⁶. 6 Ω, capacitor size is 10 - 5F. The circuit is built on a breadboard. A semiconductor parameter analyzer is then used to transmit signals to the resistor-capacitor series circuit. Quick IV Measurement Software can adjust the voltage magnitude and pulse period, and can also be used to record voltage changes. Consistent with the signal settings in the simulation, when the signal is a pixel with a value of 1 (i.e., a white pixel), a voltage of 10V is applied across the RC circuit for 0.94s in a single period of 3.29s. When the signal is a pixel with a value of 0 (i.e., a black pixel), there is no voltage input. The voltage value at the end of the fourth signal period of each row is selected as the output of the actual hardware reservoir. The weight training and testing process is consistent with the simulation experiment, such as... Figure 3 As shown.

[0051] Example 2

[0052] Further testing of the model performance was conducted using three complex image datasets: MNIST (handwritten digit set), E-MNIST (handwritten letter set), and Fashion-MNIST (clothing set). This demonstrated that PWM signal modulation can effectively utilize the physical characteristics of resistor-capacitor series circuits to achieve higher performance.

[0053] like Figure 4 As shown, preprocessing of the input image can be completed through image flattening and masking operations. This preprocessing is then converted into a PWM signal and input to a resistor-capacitor series circuit. The output signal is the response of the resistor-capacitor series circuit. The end point of each pulse cycle is selected as the virtual node output in the reservoir, ultimately achieving a recognition accuracy of 96.8%. Figure 5 As shown.

[0054] At the same time, the inventors simultaneously compared Pulse-Amplitude Modulation (PAM) with PWM, such as... Figure 6 As shown, it was found that when the reservoir size exceeds 784, the accuracy of PAM no longer increases. This is because PAM cannot truly utilize the nonlinearity of the resistor-capacitor series circuit. Combined with formula (1), for PAM, with a fixed input time, the input signal used is V. in In the case of a linear mapping, the actual mapping remains linear. PWM, on the other hand, fixes the input voltage and utilizes the t0 part, which effectively leverages the change in the exponential part of the resistor-capacitor series circuit, ultimately enabling a non-linear data mapping.

[0055] in addition, Figure 7 and Figure 8 The results of recognizing handwritten letters and clothing using the recognition method provided in this embodiment are also shown, which fully demonstrates the effectiveness and feasibility of this embodiment.

[0056] Based on the same inventive concept, a second embodiment of this disclosure provides an image recognition device based on a physical storage tank, the structural schematic of which is shown below. Figure 9 As shown, it mainly includes a model building module 10, used to build an image recognition model based on a physical reservoir; the image recognition model includes at least an input layer, a reservoir layer, and an output layer, wherein the input layer is used to preprocess the input image; the reservoir layer includes N resistor-capacitor series circuits, each resistor-capacitor series circuit corresponding to a processing channel, used to perform nonlinear mapping on the preprocessed image; the output layer is used to output the recognition result; the model training module 20 is used to train the image recognition model using labeled sample images; the image recognition module 30 is used to input the image to be recognized into the image recognition model and determine the output result of the image recognition model as the image recognition result.

[0057] The functions that the above-mentioned functional modules can achieve have been described in detail in the first embodiment of this disclosure, and will not be repeated here.

[0058] This embodiment utilizes a resistor-capacitor series circuit as the reservoir layer of the physical reservoir for image recognition, making the reservoir network structure simple and easy to build. In addition, through the nonlinear characteristics and short-time memory capability of the capacitor, the reservoir can effectively capture data dynamically, reduce the number of weights required for training, and ensure high accuracy of image recognition results.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this disclosure.

Claims

1. An image recognition method based on a physical reservoir, characterized in that, include: An image recognition model based on a physical reservoir is constructed. The image recognition model includes at least an input layer, a reservoir layer, and an output layer. The input layer is used to convert the input image into an input signal. The reservoir layer includes N resistor-capacitor series circuits, each of which corresponds to a processing channel for nonlinear mapping of the input signal. The output layer is used to output the recognition result. The image recognition model is trained using labeled sample images; The image to be recognized is input into the image recognition model, and the output result of the image recognition model is determined as the recognition result of the image; The reservoir layer performs nonlinear mapping on the preprocessed image, including: N PWM signals are input into their corresponding resistor-capacitor series circuits for short-term memory, and the end time of a preset pulse period in each PWM signal is selected as a virtual node to determine the capacitor voltage value at all virtual nodes. When the image to be identified is a simple type image, the preset pulse period is the last pulse period in the PWM signal; when the image to be identified is a complex type image, the preset pulse period is each pulse period in the PWM signal.

2. The image recognition method according to claim 1, characterized in that, The input layer is specifically used to perform any one or more of the following on the input image: image shaping, masking, and binarization.

3. The image recognition method according to claim 1 or 2, characterized in that, The images to be identified include simple type images and complex type images; wherein, the simple type images include 0 to 9 digit images composed of pixels, and the complex type images include handwritten digit images, handwritten letter images, and clothing images.

4. The image recognition method according to claim 3, characterized in that, The input signal consists of N PWM signals, where the pulse width of each PWM signal is proportional to the pixel value of the corresponding pixel in the input image.

5. The image recognition method according to claim 1, characterized in that, The capacitor voltage value at the virtual node is determined based on the following formula: in, This indicates the magnitude of the capacitor voltage value read at the virtual node; Indicates the magnitude of the input voltage; T represents the duration of a single pulse cycle; Indicates the duration of the high level within a single pulse cycle; The time parameter is determined by the hardware parameters of the resistor-capacitor circuit. R is the resistance and C is the capacitance.

6. The image recognition method according to claim 1, characterized in that, The output layer outputs the recognition results, including: Perform matrix operations on the capacitor voltage values ​​at all virtual nodes and the trained weight matrix, use a winner-takes-all strategy to determine the label value of the result, and determine the image recognition result based on the label value.

7. The image recognition method according to claim 6, characterized in that, The image recognition model is trained using labeled sample images, including training the weight matrix using linear regression.

8. An image recognition device based on a physical reservoir, characterized in that, include: A model building module is used to construct an image recognition model based on a physical reservoir. The image recognition model includes at least an input layer, a reservoir layer, and an output layer. The input layer is used to preprocess the input image. The reservoir layer includes N resistor-capacitor series circuits, each of which corresponds to a processing channel for nonlinear mapping of the preprocessed image. The output layer is used to output the recognition result. The model training module is used to train the image recognition model using labeled sample images; An image recognition module is used to input the image to be recognized into the image recognition model and determine the output result of the image recognition model as the recognition result of the image; The reservoir layer performs nonlinear mapping on the preprocessed image, including: inputting N PWM signals into their corresponding resistor-capacitor series circuits for short-term memory, selecting the end time of a preset pulse period in each PWM signal as a virtual node, and determining the capacitor voltage value at all virtual nodes; when the image to be identified is a simple type image, the preset pulse period is the last pulse period in the PWM signal; when the image to be identified is a complex type image, the preset pulse period is each pulse period in the PWM signal.

Images