Information processing method, information processing apparatus, and program

Abstract

To provide an information processing method related to a physical reservoir and on the basis of a more efficient hardware mounting method.SOLUTION: An information processing method of the present invention uses physical reservoir calculation, and the information processing method includes a step of changing the input frequency of time-series data (time-series pattern) input from a node of an input layer, a step of performing calculation on the basis of the input time-series data, and a step of outputting a result of the calculation from an output layer. In particular, the method includes changing the input frequency to a speed according to the time constant unique to the physical reservoir to be used, to extract the characteristics of an image having a specific spatial frequency. An information processing apparatus 1 of the present invention comprises a processing unit having an input layer, a reservoir layer, and an output layer. The reservoir layer is a physical reservoir. The apparatus comprises a control unit that changes the input frequency of the time-series data input from a node of the input layer.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing method, particularly an information processing method using physical reservoir computing. The present invention also relates to an information processing device, particularly a physical reservoir computing device. The present invention also relates to a program for causing a computer to operate as the information processing device. [Background technology]

[0002] Neural networks (NNs) that mimic the functions and structures of neurons in the human brain are known, as are recurrent neural networks (RNNs) that have a structure in which signals are passed back within the network (the output of a certain layer is passed back to the input). However, while current AI can achieve astonishing performance, it requires a large amount of training data to achieve high performance, and training requires enormous computational costs. Furthermore, running this on existing computers consumes a huge amount of power. Therefore, it can be difficult to operate current AI in edge areas where training data, computing resources, and power consumption are limited.

[0003] Therefore, in recent years, a machine learning framework with low learning costs called reservoir computing (RC) and its hardware implementation have been attracting attention as a solution to implement AI in edge areas to replace current AI. Against this background, the inventor has been researching an information processing method using reservoir-based convolution operations that can be learned at the edge (Patent Document 1, Non-Patent Document 3) and the physical implementation of reservoirs using nanomaterials (Patent Document 2, Non-Patent Document 4). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2023-087931 [Patent Document 2] Patent Publication No. 2023-056148 [Non-patent literature]

[0005] [Non-Patent Document 1] Y. Tanaka and H. Tamukoh, “Reservoir-based convolution,” Nonlinear Theory and Its Applications, IEICE, Vol.E13-N, No.2, 2022. [Non-patent document 2] Y. Usami et al., “In-materio reservoir computing in a sulfonated polyaniline network,” Advanced Materials, Vol.33, No.48, 2102688, 2021. Summary of the Invention [Problem to be solved by the invention]

[0006] However, due to the aforementioned background, in recent years, there has been a demand for establishing a hardware implementation method for more efficient reservoir-based convolution operations with reduced power consumption, which can achieve performance improvements such as calculation accuracy. Therefore, an object of the present invention is to provide an information processing method and the like based on a more efficient hardware implementation method for a physical reservoir. [Means for solving the problem]

[0007] The information processing method of the present invention is an information processing method using physical reservoir calculations, and includes the steps of changing the input frequency of time series data (time series patterns) input from nodes in the input layer, performing calculations based on the input time series data, and outputting the results of the calculations from the output layer.

[0008] In particular, it is desirable to extract image features having specific spatial frequencies by varying the input frequency at a rate that corresponds to the time constant specific to the physical reservoir used.

[0009] On the other hand, the information processing device of the present invention is an information processing device equipped with a processing unit having an input layer, a reservoir layer, and an output layer, wherein the reservoir layer is a physical reservoir, and the information processing device is equipped with a control unit that changes the input frequency of time series data input from a node in the input layer.

[0010] In particular, it is desirable that the control unit be configured to extract image features having a specific spatial frequency by converting the input frequency to a speed according to a time constant specific to the physical reservoir used.

[0011] The program of the present invention is a program for causing a computer to operate as the information processing device described above. [Effects of the Invention]

[0012] According to the information processing method of the present invention, the input frequency of input time series data can be changed, and therefore the input signal frequency to the device can be changed according to an appropriate time constant for appropriately processing the signal (input time series data). As a result, by selecting the optimal input signal frequency, it is possible to achieve improved performance such as calculation accuracy. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 illustrates an example of a typical reservoir computing model. [Figure 2] Figure 1 shows a reservoir-based convolution circuit system using in-material RC. [Figure 3] FIG. 1 shows physical reservoir calculations with sulfonated polyaniline networks. [Figure 4] FIG. 1 is a diagram illustrating an in-material RC device which is an information processing device according to an embodiment of the present invention. [Figure 5] FIG. 10 is a diagram illustrating an example of a response according to an input frequency. DETAILED DESCRIPTION OF THE INVENTION

[0014] The following describes in detail an embodiment of the present invention. However, the following description of the components is an example (typical example) of an embodiment of the present invention, and the present invention is not limited to the following content unless the gist of the present invention is changed. In addition, in this description, a node is synonymous with a neuron, a reservoir layer is synonymous with a reservoir, and time series data is synonymous with a time series pattern.

[0015] [Reservoir Computing] Reservoirs are suitable for classification and prediction using time series data, and require nonlinearity to perform nonlinear transformations on input data (input time series data). Reservoirs also need to store past input data (short-term memory) in order to classify and regress the input data. Known reservoir computing models, such as ESN (Echo State Network), LSM (Liquid State Machine), and CBM-RC (Chaotic Boltzmann Machine-RC), are shown in Figure 1. where N i is the number of nodes in the input layer, N r is the number of nodes in the reservoir (reservoir layer), N о is the number of nodes in the output layer. ir is the weight between the input layer and the reservoir, and W rr is the load (weight) between reservoirs, and W ro is the weight between the reservoir and the output layer. In reservoir computing, W ir and W rr is fixed by a random number. On the other hand, W ro is optimized using ridge regression etc.

[0016] [Information processing device of the present invention] An information processing device 1 of the present invention (see FIG. 4; hereinafter, the information processing device 1 according to the embodiment of the present invention will be referred to as an "in-material RC device") is a reservoir-based neural network suitable for use in image processing (particularly image recognition). In the reservoir-based convolution operation in a reservoir-based neural network, a local region of an image (Region of Interest: ROI) is regarded as a time-series pattern, and its features are extracted by processing it in a reservoir. In an in-material RC device, however, the time-series pattern is provided to a physical reservoir, and the response is used as the processing result for that local region (see FIG. 2).

[0017] That is, the in-material RC device includes a processing unit (not shown) having an input layer, a reservoir layer, a pooling layer, and an output layer (read-out layer), where the reservoir layer is a physical reservoir. Time-series data is input from the input layer (input layer nodes), physical reservoir calculations are performed in the reservoir layer, and a response is output from the output layer (see Figure 4).

[0018] The in-material RC device uses a physical reservoir made of sulfonated polyaniline network (SPAN) (see Figure 3) to implement a reservoir-based convolutional neural network. Nanomaterial reservoirs are an effective option for realizing physical reservoirs due to their simple fabrication process. In this case, electrical carriers are the driving force for the flow of electrical charges through nanomaterial networks. Furthermore, in nanomaterial networks, complex carrier transport pathways are generated along with networks that include nonlinear electrical behavior at junctions on time scales of μs to ms, which can be considered as chemical dynamics. Thus, chemical dynamics in nanomaterial networks possess rich reaction rates and spatiotemporal dynamics, enabling high-dimensional nonlinear mapping of the reservoir.

[0019] In particular, SPAN is one of the effective options for nanomaterial reservoirs. SPAN is a derivative of polyaniline, one of the most well-known conductive polymers. It forms polymer-assembled string structures with a width of approximately 5 nm, enabling the formation of dense and complex networks. Furthermore, SPAN has protonated sulfone functional groups, which act as dopants. In addition to other conductive polymers, polarons are also inherent carriers in SPAN. Furthermore, atmospheric protons are injected into SPAN molecular chains, and polarons and ions in SPAN each have different carrier mobilities. Therefore, SPAN possesses rich and complex chemical dynamics due to the dual-carrier combination.

[0020] While reservoir-based convolution extracts features with various spatial frequencies from an image by changing the reservoir's time constant, physical reservoirs have their own time constants. Therefore, in-material RC devices extract features with various spatial frequencies by adjusting the input rate of the time series pattern. Alternatively, in-material RC devices extract features with various spatial frequencies by providing a time series pattern at a constant input rate to physical reservoirs with various time constants.

[0021] The speed at which this time series pattern is applied can be changed by a control unit (not shown) included in the in-material RC device. Specifically, the control unit changes the input frequency of the time series pattern input from the nodes in the input layer within the range of 1 kHz to 50 kHz. Of course, the control unit can also apply a time series pattern at an input frequency lower than the example value (less than 1 kHz) or higher than the example value (more than 50 kHz).

[0022] [Information processing method of the present invention] Next, an information processing method (hereinafter, simply referred to as "information processing method") of the present invention using an in-material RC device will be described along with examples. In the examples, the results of a task of classifying handwritten digit images (MNIST dataset) using an in-material RC device will be shown.

[0023] (1. Construction) First, the input time series data is a three-step time series of three-dimensional vectors representing the ROI of a pixel, and the state of the output port immediately after input is used as the layer output. Since each output port corresponds to a channel of the output feature map, in this embodiment, the number of generated output feature maps is 13 (see Figures 2 and 4).

[0024] In this example, the input image value range was assumed to be [0, 1], and when the input image value was applied as a voltage to the device, the input port voltage (input voltage) range was [0V, 1V]. Meanwhile, experiments revealed that the output port voltage (output voltage) range was [-0.20V, 0.20V]. Therefore, the output voltage was multiplied by 5 to determine the value of the output feature map.

[0025] With this design, the in-material RC device was constructed as a reservoir-based CNN with one reservoir-based convolutional layer and 13 output feature maps. Therefore, a network can be constructed using one in-material RC device with 16 input / output ports (3 input ports, 13 output ports). The calculations of the pooling layer (max pooling layer) and linear layer (fully connected layer) were performed using software (see Figure 4).

[0026] (2. Input frequency of time series data) Next, we investigated the input frequency. In-material RC devices (physical reservoirs) have an appropriate time constant for signal processing, so the input signal frequency to the device is thought to cause changes in the RC state. For example, if the input signal frequency is too low for the device, the effect of each input to the reservoir disappears in the next time step, and the reservoir state does not represent the entire characteristics of the input time series data. Conversely, if the input signal frequency is too high for the device, the changes in the reservoir state cannot keep up with the input changes (changes in the input time series data), and the reservoir cannot fully process the input time series data.

[0027] For these reasons, the inventors investigated an appropriate input frequency for the in-material RC device. In this investigation, the input frequency was changed from 1 kHz to 50 kHz while checking the response of the in-material RC device. Images from the MNIST dataset were then input to the in-material RC device over this input frequency range.

[0028] Figure 5 shows feature maps generated from the in-material RC device with input frequencies set to (A) 1 kHz, (B) 5 kHz, (C) 10 kHz, and (D) 50 kHz. The results show that when the input frequency was 1KHz or 5KHz, the line edges were extracted and the feature map had some variance. Conversely, when the input frequency was 50KHz, the generated feature map was consistent (almost no variance) and only slightly different from the input image. When the input frequency was 10KHz, the generated feature map had some variance, but the line edges could not be accurately extracted.

[0029] Therefore, for in-material RC devices implemented using SPAN, 1 kHz to 5 kHz is an appropriate input frequency.

[0030] (3. Image Classification) Finally, to verify the performance of the in-material RC device, we performed an image classification task using the MNIST dataset. The MNIST dataset used consists of 60,000 training images and 10,000 test images, from which 1,000 training images and 1,000 test images (with no overlap with the training images) were randomly selected and used for this verification.

[0031] The verification procedure is as follows: (1) The training image was input to the in-material RC device, and the optimized synaptic weight connections of the linear layer were calculated using ridge regression using both the output of the max pooling layer and the teacher signal, which was a one-hot vector corresponding to the class label. (2) Then, test images were input into the in-material RC device to verify its accuracy. The coefficient of the regularization term of the ridge regression was set to 1.0, and the input frequency was set to 1 kHz.

[0032] Table 1 compares the accuracy of [1] the linear model, [2] the reservoir-based CNN with ESN implementation, and [3] the accuracy of the in-material RC device. Here, the weight connections of the ESN were initialized randomly, so the accuracy of [2] the reservoir-based CNN with ESN implementation in Table 1 is the average of 10 trials.

[0033] [Table 1]

[0034] Table 1 shows that the in-material RC device (81.7%) is effective, as it shows better results than the linear model (74.4%). On the other hand, the accuracy of the in-material RC device is inferior to that of the reservoir-based CNN (approximately 87.7%) implemented in the ESN. This is attributed to the noise in the generated feature map, as shown in Figure 5(A).

[0035] In the image classification task, the input frequency was set to 1 kHz as described above, and it took 3 ms to calculate a 3 × 3 ROI. Therefore, when using one in-material RC device as in this embodiment, the ROI calculation time is estimated to be 3 ms. To speed up this computation, the reservoir-based convolution can be implemented in parallel using multiple in-material RC devices. Furthermore, increasing the input frequency can speed up the computation, but this may result in a loss of accuracy, which requires further verification.

[0036] Although the in-material RC device has a small memory capacity, the reservoir only receives a 3 × 3 ROI converted into a 3-step time series of 3D vectors, so it does not require a large memory. Therefore, the in-material RC device is suitable for reservoir-based convolution from the viewpoint of device characteristics. Furthermore, in the present invention, the calculation processes required for implementation using existing computers are directly replaced by the dynamics of the physical reservoir, which is expected to result in a significant reduction in power consumption.

[0037] The information processing device (in-material RC device) and information processing method described above are merely examples of one embodiment, and the design can be modified as appropriate as long as the gist of the device is not changed. For example, as described above, multiple in-material RC devices can be used, or the number of output ports can be changed.

[0038] Furthermore, although the present embodiment uses a sulfonated polyaniline network as a physical reservoir, the physical reservoir may also be a physical reservoir made of silver selenide, a physical reservoir made of carbon nanotubes, an optical reservoir, etc. Each of these has its own time constant, and the time constant also varies depending on the size of the device. Therefore, the present invention can change the input signal frequency to the device depending on an appropriate time constant that allows the signal to be processed appropriately. Therefore, the configuration of the present invention may be configured such that, for example, the relationship between the time constant and the corresponding input signal frequency is stored in a storage unit such as a memory or database, and the control unit refers to this and appropriately changes the input signal frequency to a speed that corresponds to the time constant specific to the physical reservoir used, thereby making it possible to extract features of an image having a specific spatial frequency.

[0039] Additionally, in this embodiment, the pooling layer and linear layer are software-based rather than implemented on a circuit, but these layers may be implemented on an integrated circuit to achieve low power consumption. [Industrial Applicability]

[0040] The present invention relates to a physical reservoir and provides an information processing method, an information processing device, and a program based on an efficient hardware implementation method, which are applicable to many physical reservoirs and are therefore industrially useful. [Explanation of symbols]

[0041] 1. Information processing device (In-material RC device)

Claims

1. An information processing method using physical reservoir computation, comprising: changing the input frequency of time series data input from nodes in the input layer; performing a calculation based on the input time series data; outputting the result of the operation from an output layer; An information processing method including:

2. 2. The information processing method according to claim 1, wherein the input frequency is changed to a speed corresponding to a time constant specific to the physical reservoir used, thereby extracting image features having a specific spatial frequency.

3. An information processing device including a processing unit having an input layer, a reservoir layer, and an output layer, the reservoir layer is a physical reservoir; An information processing device comprising a control unit that changes the input frequency of time-series data input from the nodes of the input layer.

4. The information processing device according to claim 3 , wherein the control unit converts the input frequency into a speed according to a time constant specific to a physical reservoir used, thereby extracting a feature of an image having a specific spatial frequency.

5. Computer, An information processing device including a processing unit having an input layer, a reservoir layer, and an output layer, the reservoir layer is a physical reservoir; A program for operating an information processing device having a control unit that changes the input frequency of time-series data input from the nodes of the input layer.

Images