Storage pool operation method and electronic equipment based on photoelectric devices

Abstract

The application discloses a storage pool operation method based on an optoelectronic device and electronic equipment, relates to the fields of machine learning and artificial intelligence, and is applied to an adaptive delay storage network carried in an optical device. The adaptive delay storage network comprises a neural network unit and a horizontal modulation network unit. The horizontal modulation network unit comprises, in sequence, a photocurrent input layer module, a first full connection layer module, a second full connection layer module and a modulation gate voltage output layer module. The photocurrent input layer module receives a virtual node state output in the form of a response current by a storage pool layer module. The first full connection layer module and the second full connection layer module perform data processing on a plurality of response currents, and determine current feature vector information corresponding to the plurality of response currents. The modulation gate voltage output layer module converts the response current information into a modulation gate voltage value by a mapping function, and transmits the modulation gate voltage value to the storage pool layer module, so that target optoelectronic response output information is obtained under the corresponding modulation gate voltage value.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine learning and artificial intelligence, and in particular to a storage pool operation method based on an optoelectronic device and an electronic device. BACKGROUND

[0002] With the large application of autonomous vehicles, smart homes, video surveillance and other scenarios, edge artificial vision systems applied to the Internet of Things have become a research field that has attracted much attention. However, the traditional vision system is mostly composed of separate sensors, memories and processors, and the frequent data transfer leads to high latency and power consumption.

[0003] To solve the above problems, neuromorphic vision systems integrating sensing, memory and processing functions have been widely researched and developed. However, in terms of hardware, although the new optoelectronic sensors in the neuromorphic vision system can exhibit high sensitivity and flexibility, they are mostly used to implement basic image preprocessing functions. In this image preprocessing process, a relatively long information processing time is usually required, and additional computing networks are relied on to support more complex functions, thereby introducing additional system latency and complexity. In terms of software, the storage pool computing paradigm widely used at present is weakened in flexibility of network structure and limited in application in complex actual environment due to the fixed physical mechanism and dynamic response of devices in the hardware implementation process. Therefore, the existing neuromorphic vision system has low information processing efficiency, high computing cost, small use range, poor stability and reliability. SUMMARY

[0004] The present application aims to provide a storage pool operation method based on an optoelectronic device and an electronic device to solve the problems of low information processing efficiency, high computing cost, small use range, poor stability and reliability of the existing neuromorphic vision system.

[0005] In a first aspect, the present application provides a storage pool operation method based on an optoelectronic device, applied to an adaptive delay storage network carried by an optical device, wherein the adaptive delay storage network comprises a neural network unit and a horizontal modulation network unit; the neural network unit comprises a storage pool layer module, and the horizontal modulation network unit comprises an optocurrent input layer module, a first full connection layer module, a second full connection layer module and a modulation gate voltage output layer module connected in sequence; wherein the storage pool layer module is connected with the horizontal modulation network unit, and the method comprises:

[0006] The optocurrent input layer module receives the virtual node state output in the form of response current by the storage pool layer module;

[0007] The first fully connected layer module and the second fully connected layer module perform data processing on the plurality of response currents to determine current feature vector information corresponding to the plurality of response currents;

[0008] The modulated gate voltage output layer module converts the multiple response current information into modulated gate voltage values ​​through a mapping function, and transmits the modulated gate voltage values ​​to the storage pool layer module so that the neural network unit can control the photoelectric device to obtain the target photoelectric response output information under the corresponding modulated gate voltage values.

[0009] When the above technical solution is adopted, it is applied to the adaptive delay storage network carried by the optical device, and the adaptive delay storage network includes a neural network unit and a horizontal modulation network unit; the neural network unit includes a storage pool layer module, and the horizontal modulation network unit includes a photocurrent input layer module, a first fully connected layer module, a second fully connected layer module and a modulated gate voltage output layer module connected in sequence; wherein, the storage pool layer module and the horizontal modulation network unit are connected, and the method includes: the photocurrent input layer module receives the virtual node state output by the storage pool layer module in the form of a response current; the first fully connected layer module and the second fully connected layer module perform data processing on a plurality of the response currents to determine The current characteristic vector information corresponding to the multiple response currents; the modulation gate voltage output layer module converts the multiple response current information into modulation gate voltage values ​​by a mapping function, and transmits the modulation gate voltage values ​​to the storage pool layer module, so that the neural network unit controls the photoelectric device to obtain the target photoelectric response output information under the corresponding modulation gate voltage value, which can realize visual adaptation and feature extraction functions at the sensing end at the same time, and combines the advantages of photoelectric sensor devices and storage pool calculations to achieve network performance optimization through dynamic modulation. At the same time, by adding trainable horizontal modulation network units, the system can realize real-time autonomous dynamic modulation, which provides the possibility for realizing efficient, low computing cost, wide application range and stable performance edge vision system.

[0010] In a possible implementation, the photocurrent input layer module receives the virtual node state output by the storage pool layer module in the form of a response current, including:

[0011] The photocurrent input layer module receives the virtual node state output by the storage pool layer module in the form of a response current after the optoelectronic device of the storage pool layer module receives the light pulse and generates an original accumulation response.

[0012] In a possible implementation, the virtual node state is a response current at different times on a delay line generated by an array structure in the optoelectronic device in response to a light pulse.

[0013] In a possible implementation, the first fully connected layer module and the second fully connected layer module perform data processing on the plurality of response currents to determine current feature vector information corresponding to the plurality of response currents, including:

[0014] The first fully connected layer module compresses the plurality of response currents based on the first weight matrix and in combination with a linear rectification activation function;

[0015] The second fully connected layer module performs dimensionality reduction processing on the current feature vector based on the second weight matrix in combination with a hyperbolic tangent activation function to determine current feature vector information corresponding to multiple response currents.

[0016] In a possible implementation, the value ranges of the first weight matrix and the second weight matrix respectively satisfy:

[0017] Wherein, W1 represents the first weight matrix, W2 represents the second weight matrix; R represents a set of real numbers, r represents a compression parameter, and C represents the number of channels.

[0018] In a possible implementation, the current characteristic vector information is expressed as:

[0019] G=F HM (I,W)=σ(W2δ(W1I));

[0020] Wherein, G represents the current characteristic vector information, and F HM represents the calculation function of the horizontal modulation network unit; I represents the current eigenvector, and W represents the weight matrix.

[0021] In a possible implementation, before the photocurrent input layer receives the virtual node state output by the storage pool layer module in the form of a response current, the method further includes:

[0022] When training the adaptive delay storage network, performing a forward propagation;

[0023] Determine the network weight error value based on the cross entropy function;

[0024] The gradients of the weights of the neural network unit and the horizontal modulation network unit are determined and updated based on the network weight error value through back propagation.

[0025] In one possible implementation, the neural network unit further includes an input layer module and a fully connected layer module respectively connected to the storage pool layer module, and an output layer module connected to the fully connected layer module; the modulated gate voltage output layer module transmits the modulated gate voltage value to the storage pool layer module so that the neural network unit controls the photoelectric device to obtain target photoelectric response output information under the corresponding modulated gate voltage value, including:

[0026] The modulated gate voltage output layer module transmits the modulated gate voltage value to the storage pool layer module so that the fully connected layer and the output layer can control the photoelectric device to obtain target photoelectric response output information under the corresponding modulated gate voltage value.

[0027] In a possible implementation, the light pulse received by the optoelectronic device is a mixed light intensity MNIST data set.

[0028] In a second aspect, the present application also provides an electronic device comprising: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, enables the storage pool operation method based on optoelectronic devices described in any possible implementation of the first aspect to be executed.

[0029] The beneficial effects of the electronic device provided by the second aspect are the same as the beneficial effects of the storage pool operation method based on optoelectronic devices described in the first aspect or any possible implementation of the first aspect, and are not repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0031] Figure 1 A schematic flow chart of a storage pool operation method based on a photoelectric device provided in an embodiment of the present application is shown;

[0032] Figure 2 A schematic diagram of the structure of an adaptive delay storage network provided by an embodiment of the present application is shown;

[0033] Figure 3 A schematic flow chart of another storage pool operation method based on a photoelectric device provided in an embodiment of the present application is shown;

[0034] Figure 4 A schematic diagram of a data set provided in an embodiment of the present application is shown;

[0035] Figure 5The following is a general flow chart of implementing network adaptation using a horizontal modulation network according to an embodiment of the present application;

[0036] Figure 6 A flowchart of training an adaptive delay storage network provided by an embodiment of the present application is shown;

[0037] Figure 7 A schematic diagram showing the effect of a change in device gate voltage on recognition rate under different light intensities provided by an embodiment of the present application;

[0038] Figure 8 A schematic diagram showing an original storage pool algorithm network, recognition accuracy under small-scale autonomous gate voltage modulation and large-scale autonomous gate voltage modulation conditions provided by an embodiment of the present application is shown;

[0039] Figure 9 A schematic diagram of a network training process under the conditions of an original storage pool algorithm network, small-scale autonomous gate voltage modulation, and large-scale autonomous gate voltage modulation provided by an embodiment of the present application is shown;

[0040] Figure 10 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown;

[0041] Figure 11 It is a schematic diagram of the structure of the chip provided in the embodiment of the present application. DETAILED DESCRIPTION

[0042] To facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. For example, the first threshold and the second threshold are merely used to distinguish between different thresholds and do not limit their order. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity or execution order, and words such as "first" and "second" do not necessarily mean that they are different.

[0043] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0044] In this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, the combination of a and b, the combination of a and c, the combination of b and c, or the combination of a, b and c, where a, b, c can be single or multiple.

[0045] Figure 1 The figure shows a flow chart of a storage pool operation method based on an optoelectronic device provided in an embodiment of the present application, which is applied to an adaptive delay storage network carried by an optical device. Figure 2 FIG. 1 shows a schematic diagram of a structure of an adaptive delay storage network provided by an embodiment of the present application. Figure 2 As shown, the adaptive delay storage network includes a neural network unit 101 and a horizontal modulation network unit 102; the neural network unit 101 includes a storage pool layer module 101A, and the horizontal modulation network unit 102 includes a photocurrent input layer module 102A, a first fully connected layer module 102B, a second fully connected layer module 102C and a modulation gate voltage output layer module 102D connected in sequence; wherein, the storage pool layer module 101A and the horizontal modulation network unit 102 are connected.

[0046] In this application, the main body of the adaptive delay storage network is a delay storage pool composed of a MoS2 (molybdenum disulfide) device array with photoresponsive electrical modulation and short-term memory characteristics. The number of channels of the delay storage pool is C = 28. At the same time, a horizontal modulation network unit consisting of two fully connected layers (a first fully connected layer module and a second fully connected layer module) is added to realize the adaptive process. In this application, the network size of the photocurrent input layer module-the first fully connected layer module-the second fully connected layer module is 28-1-28, that is, the photocurrent input layer module has 28 nodes, the first fully connected layer module has 1 node, and the second fully connected layer module has 28 nodes.

[0047] like Figure 1 As shown, the method includes:

[0048] Step 201: The photocurrent input layer module receives the virtual node status output by the storage pool layer module in the form of a response current.

[0049] In the present application, the photocurrent input layer module receives the virtual node state output by the storage tank layer module in the form of a response current after the optoelectronic device of the storage tank layer module receives a light pulse and generates a primitive accumulation response.

[0050] Step 202: The first fully connected layer module and the second fully connected layer module perform data processing on the multiple response currents to determine current feature vector information corresponding to the multiple response currents.

[0051] In the present application, the first fully connected layer module compresses the multiple response currents based on the first weight matrix and combines the linear rectification activation function, and the second fully connected layer module performs dimensionality reduction processing on the current feature vector based on the second weight matrix and combines the hyperbolic tangent activation function to determine the current feature vector information corresponding to the multiple response currents.

[0052] Step 203: The modulated gate voltage output layer module converts the multiple response current information into modulated gate voltage values ​​through a mapping function, and transmits the modulated gate voltage values ​​to the storage pool layer module so that the neural network unit can control the photoelectric device to obtain the target photoelectric response output information under the corresponding modulated gate voltage values.

[0053] The storage pool operation method based on optoelectronic devices provided in the embodiment of the present application is applied to an adaptive delay storage network carried by an optical device, wherein the adaptive delay storage network includes a neural network unit and a horizontal modulation network unit; the neural network unit includes a storage pool layer module, and the horizontal modulation network unit includes a photocurrent input layer module, a first fully connected layer module, a second fully connected layer module and a modulated gate voltage output layer module connected in sequence; wherein the storage pool layer module and the horizontal modulation network unit are connected, and the method includes: the photocurrent input layer module receives the virtual node state output by the storage pool layer module in the form of a response current; the first fully connected layer module and the second fully connected layer module perform digital processing on a plurality of the response currents According to the processing, the current characteristic vector information corresponding to the multiple response currents is determined; the modulation gate voltage output layer module converts the multiple response current information into modulation gate voltage values ​​by the mapping function, and transmits the modulation gate voltage values ​​to the storage pool layer module, so that the neural network unit can control the photoelectric device to obtain the target photoelectric response output information under the corresponding modulation gate voltage value, and can realize visual adaptation and feature extraction functions at the sensing end at the same time. Combining the advantages of photoelectric sensor devices and storage pool calculations, the network performance is optimized through dynamic modulation. At the same time, by adding trainable horizontal modulation network units, the system can realize real-time autonomous dynamic modulation, which provides the possibility for realizing efficient, low computing cost, wide application range and stable performance edge vision system.

[0054] Figure 3 FIG. 1 shows a flow chart of another storage pool operation method based on a photoelectric device provided in an embodiment of the present application. Figure 3 As shown, the method includes:

[0055] Step 301: The photocurrent input layer module receives the virtual node status output by the storage pool layer module in the form of response current.

[0056] In the present application, the photocurrent input layer module receives the virtual node state output by the storage tank layer module in the form of a response current after the optoelectronic device of the storage tank layer module receives a light pulse and generates a primitive accumulation response.

[0057] The virtual node state is the response current at different times on the delay line generated by the array structure in the optoelectronic device in response to the light pulse.

[0058] Specifically, in the initial state, the array of photovoltaic devices is applied with the same initial gate voltage When a light pulse input is applied to the device, a nonlinear transient response is obtained. Since the device has a short-term memory function, a characteristic vector I containing time and space information can be constructed by reading the response current at different times on the delay line, where the delay line represents the decay process of the current.

[0059] Among them, the initial gate voltage can be 1V, which is not specifically limited in the embodiment of the present application and can be specifically adjusted according to the actual application scenario.

[0060] In this application, the light pulses received by the optoelectronic device are mixed light intensity MNIST data sets, Figure 4 A schematic diagram of a data set provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the data set of this application adopts the mixed light intensity MNIST data set, and the adaptive delay storage network of this application needs to complete the image recognition task under different light intensities. Figure 4 As shown in Figure 1, the MNIST dataset is divided into two subsets, which are encoded as strong light input L1 and weak light input L2. After encoding, the subtask images are re-merged and shuffled for the adaptive recognition task to obtain the mixed light intensity MNIST dataset.

[0061] Step 302: The first fully connected layer module compresses the multiple response currents based on the first weight matrix and in combination with a linear rectification activation function.

[0062] In this application, network modulation and adaptive recognition can be achieved through modulation network and feedback loop, Figure 5 FIG. 4 shows an overall flow chart of a method for realizing network adaptation by using a horizontal modulation network according to an embodiment of the present application. Figure 5As shown, at each virtual node, the response current I C The input level modulation network unit obtains a network output G corresponding to each channel. The first fully connected layer module compresses the multiple response currents, specifically, based on the first weight matrix W1 combined with the linear rectifier activation function.

[0063] Specifically, the data can be nonlinearly transformed using the Sigmoid activation function.

[0064] Step 303: The second fully connected layer module performs dimensionality reduction processing on the current feature vector based on the second weight matrix and in combination with the hyperbolic tangent activation function to determine the current feature vector information corresponding to the multiple response currents.

[0065] In this application, the value ranges of the first weight matrix and the second weight matrix respectively satisfy:

[0066] Wherein, W1 represents the first weight matrix, W2 represents the second weight matrix; R represents a set of real numbers, r represents a compression parameter, and C represents the number of channels.

[0067] By G=F HM (I, W) = σ(W2δ(W1I)) to determine the current characteristic vector information corresponding to the plurality of response currents;

[0068] Wherein, G represents the current characteristic vector information, and F HM represents the calculation function of the horizontal modulation network unit; I represents the current eigenvector, and W represents the weight matrix.

[0069] The linear rectification activation function is also the ReLU activation function, and the hyperbolic tangent activation function is also the Tanh activation function.

[0070] Step 304: The modulated gate voltage output layer module converts the multiple response current information into modulated gate voltage values ​​through a mapping function, and transmits the modulated gate voltage values ​​to the storage pool layer module so that the neural network unit can control the photoelectric device to obtain the target photoelectric response output information under the corresponding modulated gate voltage values.

[0071] In this application, the network output can be calculated by mapping function to obtain the corresponding gate voltage value of each channel Here the mapping function F map The mapping function can be set independently according to actual conditions, and the modulation voltage range and modulation granularity can be changed. Specifically,

[0072] See also Figure 2The neural network unit 101 also includes an input layer module 101B and a fully connected layer module 101C respectively connected to the storage pool layer module 101A, and an output layer module 101D connected to the fully connected layer module 101C.

[0073] The modulated gate voltage output layer module transmits the modulated gate voltage value to the storage pool layer module so that the fully connected layer and the output layer can control the photoelectric device to obtain target photoelectric response output information under the corresponding modulated gate voltage value.

[0074] In this application, the calculated modulation gate voltage can be applied to the optoelectronic device corresponding to each channel. The photoelectric dynamic characteristics of the device can be changed, thereby realizing dynamic adjustment of the output virtual node value. The storage pool state vector I obtained after modulation is input to the fully connected layer and the output layer, and the recognition output can be obtained by weighted summation, and finally the target photoelectric response output information is obtained.

[0075] In this application, the adaptive delay storage network can be trained. Figure 6 FIG. 4 shows a flow chart of training an adaptive delay storage network provided by an embodiment of the present application. Figure 6 Shown, including:

[0076] Step 401: When training the adaptive delay storage network, perform a forward propagation.

[0077] In this application, when training the adaptive delay storage network, one forward propagation can be completed.

[0078] Step 402: Determine a network weight error value based on a cross entropy function.

[0079] After completing a forward propagation, the cross entropy function can be used to calculate the network weight error.

[0080] Step 403: Determine and update the gradients of the weights of the neural network unit and the horizontal modulation network unit based on the network weight error value through back propagation.

[0081] In this application, the back propagation method can be used to simultaneously complete the gradient calculation and update of the classification layer and the horizontal modulation network weights.

[0082] For example, Figure 7A diagram showing the influence of the change of the device gate voltage on the recognition rate under different light intensities is shown, the horizontal axis represents the device gate voltage, and the vertical axis represents the recognition accuracy, different gate voltages under weak light and strong light correspond to different recognition rates, under different gate voltages, the device dynamics will change, thereby affecting the feature extraction capability of the delay storage pool, in the present application, through point modulation, a relatively high and stable recognition rate can be maintained under different light intensities.

[0083] Figure 8 A diagram showing the recognition accuracy of the original storage pool algorithm network, small-range autonomous gate voltage modulation and large autonomous gate voltage modulation is shown, under the mixed light intensity MNIST data set, the original storage pool network P1 without horizontal modulation network can only achieve a recognition accuracy of 76.43%. After adding the horizontal modulation network, the system adaptive function is realized through network training, and the recognition accuracy is greatly improved. At the same time, the larger the modulation range, the higher the accuracy. For small-range modulation P2, the recognition accuracy is 89.15%, and for large-range modulation P3, the recognition accuracy is 90.64%. The simulation results verify the effectiveness and flexibility of the network optimization of the present application.

[0084] Figure 9 A diagram showing the network training process of the original storage pool algorithm network, small-range autonomous gate voltage modulation and large autonomous gate voltage modulation is shown, the horizontal axis represents the training round of the network, and the vertical axis represents the recognition accuracy, the original storage pool network P1 after network training, the recognition accuracy on the test set decreases, and the network performance cannot be improved. After adding the horizontal modulation network, the network can learn the calculation method of the modulation gate voltage through training, thereby increasing the adaptability of the network to different tasks and the application range of the model, which can include small-range modulation P2 and large-range modulation P3.

[0085] The storage pool operation method based on the optoelectronic device provided by the embodiments of the present application can realize visual adaptation and feature extraction functions at the same time, and combines the advantages of the optoelectronic sensor device and the storage pool calculation to realize the optimization of the network performance through dynamic modulation. At the same time, through the addition of the trainable horizontal modulation network, the real-time autonomous dynamic modulation of the system is realized, which provides the possibility for realizing an efficient, low-computing-cost, wide-application-range and stable-performance edge vision system.

[0086] The electronic device in the embodiments of the present application may be a device, or a component, integrated circuit, or chip in a terminal. The device may be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device may be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc., and the embodiments of the present application do not specifically limit this.

[0087] The electronic device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0088] Figure 10 FIG1 shows a hardware structure diagram of an electronic device provided by an embodiment of the present application. Figure 10 As shown, the electronic device 500 includes a processor 510 .

[0089] like Figure 9 As shown, the processor 510 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0090] like Figure 10 As shown, the electronic device 500 may further include a communication circuit 540. The communication circuit 540 may include a path for transmitting information between the components.

[0091] Optional, such as Figure 10 As shown, the electronic device may further include a communication interface 520. There may be one or more communication interfaces 520. The communication interface 520 may be any transceiver or similar device for communicating with other devices or a communication network.

[0092] Optional, such as Figure 10As shown, the electronic device may further include a memory 530. The memory 530 is used to store computer-executable instructions for executing the solution of the present application, and the execution is controlled by the processor. The processor is used to execute the computer-executable instructions stored in the memory, thereby implementing the method provided by the embodiment of the present application.

[0093] like Figure 10 As shown, the memory 530 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 530 can exist independently and be connected to the processor 510 via a communication line 540. The memory 530 can also be integrated with the processor 510.

[0094] Optionally, the computer-executable instructions in the embodiments of the present application may also be referred to as application code, which is not specifically limited in the embodiments of the present application.

[0095] In a specific implementation, as an embodiment, Figure 10 As shown, the processor 510 may include one or more CPUs, such as Figure 10 CPU0 and CPU1 in.

[0096] In a specific implementation, as an embodiment, Figure 10 As shown, the terminal device may include multiple processors, such as Figure 10 The first processor 5101 and the second processor 5102 in the embodiment of the present invention are shown in FIG. Each of these processors can be a single-core processor or a multi-core processor.

[0097] Figure 11 This is a schematic diagram of the structure of the chip provided in the embodiment of the present application. Figure 11 As shown, the chip 600 includes one or more (including two) processors 510 .

[0098] Optional, such as Figure 11As shown, the chip also includes a communication interface 520 and a memory 530. The memory 530 may include a read-only memory and a random access memory, and provides operation instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory (NVRAM).

[0099] In some embodiments, as Figure 11 As shown, the memory 530 stores the following elements, execution modules or data structures, or a subset thereof, or an extended set thereof.

[0100] In the embodiments of this application, Figure 11 As shown, corresponding operations are performed by calling an operation instruction stored in a memory (the operation instruction may be stored in an operating system).

[0101] like Figure 11 As shown, the processor 510 controls the processing operations of any one of the terminal devices. The processor 510 may also be referred to as a central processing unit (CPU).

[0102] like Figure 11 As shown, the memory 530 may include a read-only memory and a random access memory, and provides instructions and data to the processor. A portion of the memory 530 may also include NVRAM. For example, in an application, the memory, the communication interface, and the memory are coupled together through a bus system, wherein the bus system may include a power bus, a control bus, and a status signal bus in addition to a data bus. However, for the sake of clarity, the following are not used in the following text: Figure 11 Various buses are labeled as bus system 610 .

[0103] like Figure 11As shown, the methods disclosed in the above embodiments of the present application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor or by software instructions. The above processor may be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0104] On the one hand, a computer-readable storage medium is provided, in which instructions are stored. When the instructions are executed, the functions performed by the terminal device in the above embodiment are implemented.

[0105] On the one hand, a chip is provided, which is used in a terminal device. The chip includes at least one processor and a communication interface. The communication interface is coupled to at least one processor, and the processor is used to run instructions to implement the functions performed by the storage pool operation method based on optoelectronic devices in the above embodiment.

[0106] In the above embodiments, they can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is performed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a terminal, a user device or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive (SSD).

[0107] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0108] Although the present application has been described with reference to specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely illustrative of the present application as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, the present application is intended to include such modifications and variations as fall within the scope of the claims of the present application and their equivalents.

Claims

1. A storage pool operation method based on optoelectronic devices, characterized in that: The method is applied to an adaptive delay storage network carried by an optical device, wherein the adaptive delay storage network includes a neural network unit and a horizontal modulation network unit; the neural network unit includes a storage pool layer module, and the horizontal modulation network unit includes a photocurrent input layer module, a first fully connected layer module, a second fully connected layer module, and a modulated gate voltage output layer module connected in sequence; wherein the storage pool layer module and the horizontal modulation network unit are connected, and the method includes: The photocurrent input layer module receives the virtual node state output by the storage pool layer module in the form of response current; The first fully connected layer module and the second fully connected layer module perform data processing on the plurality of response currents to determine current feature vector information corresponding to the plurality of response currents; The modulated gate voltage output layer module converts the multiple response current information into modulated gate voltage values ​​through a mapping function, and transmits the modulated gate voltage values ​​to the storage pool layer module so that the neural network unit can control the photoelectric device to obtain the target photoelectric response output information under the corresponding modulated gate voltage values.

2. The storage pool operation method based on optoelectronic devices according to claim 1, characterized in that: The photocurrent input layer module receives the virtual node state output by the storage pool layer module in the form of a response current, including: The photocurrent input layer module receives the virtual node state output by the storage pool layer module in the form of a response current after the optoelectronic device of the storage pool layer module receives the light pulse and generates an original accumulation response.

3. The storage pool operation method based on optoelectronic devices according to claim 1, characterized in that: The virtual node state is the response current at different times on the delay line generated by the array structure in the optoelectronic device in response to the light pulse.

4. The storage pool operation method based on optoelectronic devices according to claim 1, characterized in that: The first fully connected layer module and the second fully connected layer module perform data processing on the plurality of response currents to determine current feature vector information corresponding to the plurality of response currents, including: The first fully connected layer module compresses the plurality of response currents based on a first weight matrix and in combination with a linear rectification activation function; The second fully connected layer module performs dimensionality reduction processing on the current feature vector based on the second weight matrix in combination with a hyperbolic tangent activation function to determine current feature vector information corresponding to multiple response currents.

5. The storage pool operation method based on optoelectronic devices according to claim 4, characterized in that: The value ranges of the first weight matrix and the second weight matrix respectively satisfy: Wherein, W1 represents the first weight matrix, W2 represents the second weight matrix; R represents a set of real numbers, r represents a compression parameter, and C represents the number of channels.

6. The storage pool operation method based on optoelectronic devices according to claim 5, characterized in that: The current characteristic vector information is expressed as: G=F HM (I,W)=σ(W2δ(W1I)); Wherein, G represents the current characteristic vector information, and F HM represents the calculation function of the horizontal modulation network unit; I represents the current eigenvector, and W represents the weight matrix.

7. The storage pool operation method based on optoelectronic devices according to claim 1, characterized in that: Before the photocurrent input layer receives the virtual node state output by the storage pool layer module in the form of a response current, the method further includes: When training the adaptive delay storage network, performing a forward propagation; Determine the network weight error value based on the cross entropy function; The gradients of the weights of the neural network unit and the horizontal modulation network unit are determined and updated based on the network weight error value through back propagation.

8. The storage pool operation method based on optoelectronic devices according to claim 1, characterized in that: The neural network unit further includes an input layer module and a fully connected layer module respectively connected to the storage pool layer module, and an output layer module connected to the fully connected layer module; the modulated gate voltage output layer module transmits the modulated gate voltage value to the storage pool layer module so that the neural network unit controls the photoelectric device to obtain target photoelectric response output information under the corresponding modulated gate voltage value, including: The modulated gate voltage output layer module transmits the modulated gate voltage value to the storage pool layer module so that the fully connected layer and the output layer can control the photoelectric device to obtain target photoelectric response output information under the corresponding modulated gate voltage value.

9. The storage pool operation method based on optoelectronic devices according to claim 2, characterized in that: The light pulses received by the optoelectronic device are a mixed light intensity MNIST data set.

10. An electronic device, characterized in that: include: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, enable the execution of the storage pool operation method based on optoelectronic devices according to any one of claims 1 to 9.

Images