Sensing internal reservoir computing system and reconfigurable phototransistor thereof

Abstract

The invention discloses a sensing internal reservoir computing system and a reconfigurable phototransistor thereof, and relates to the field of semiconductors, and the sensing internal reservoir computing system comprises a substrate layer, a channel layer, a gate dielectric layer and an electrode layer, the channel layer is arranged on the substrate layer; the electrode layer comprises a source electrode and a drain electrode, the source electrode and the drain electrode are arranged on the channel layer at intervals, the gate dielectric layer is arranged on the channel layer between the source electrode and the drain electrode, and the gate dielectric layer serves as a gate; wherein the channel layer is an indium gallium zinc oxide thin film layer, and the gate dielectric layer is a lithium-based solid polymer electrolyte thin film layer. Dynamic regulation and control of volatility and non-volatility under light and / or electrical stimulation are achieved in a single device, and the technical problems that in an existing reserve pool computing system, the reserve state richness and stability of a reserve layer are insufficient, and the response mode is single are solved.

Description

Technical Field

[0001] This invention relates to the field of semiconductor device technology, and in particular to a sensor-internal reservoir computing system and its reconfigurable phototransistor. Background Technology

[0002] The edge deployment of artificial intelligence has driven the development of compact, energy-efficient intelligent information processing systems that integrate sensing, memory, and multitasking capabilities. However, traditional vision systems suffer from significant energy / time overhead and excessive hardware costs on edge devices, and their algorithms are often prohibitively expensive.

[0003] For example, traditional von Neumann architecture-based artificial vision systems suffer from inefficiency due to the physical separation of sensing, storage, and computing modules. This leads to frequent data transfers between these independent modules, resulting in significant energy, time, and hardware losses. Therefore, integrating sensing, storage, and computing units into a single device to achieve multi-level sensing task processing—the so-called "in-sensor computing"—holds immense potential for improving information processing efficiency and simplifying hardware architecture, despite ongoing challenges.

[0004] In-sensor pooling computation is an efficient visual information processing paradigm suitable for handling both static and dynamic time-sensitive tasks. Its main advantage lies in significantly reducing computational overhead during training (only the output network needs to be trained, while the reservoir layer does not). A typical reservoiring computation system consists of a volatile reservoir layer containing dynamically nonlinear nodes and a non-volatile readout layer network that performs a linearly weighted summation of the reservoir layer's output values. Given their distinct switching properties—volatile and non-volatile—fully integrating an in-sensor pooling computation system into a reconfigurable device remains quite challenging. Summary of the Invention

[0005] To address the technical problems existing in the background art, this invention proposes a sensor-internal reservoir computing system and its reconfigurable phototransistor.

[0006] In a first aspect, the present invention provides a reconfigurable phototransistor comprising: a substrate layer, a channel layer, a gate dielectric layer, and an electrode layer; the channel layer is disposed on the substrate layer; the electrode layer comprises a source electrode and a drain electrode, the source electrode and the drain electrode being disposed alternately on the channel layer; the gate dielectric layer is disposed on the channel layer between the source electrode and the drain electrode, and the gate dielectric layer serves as the gate electrode. The channel layer is an indium gallium zinc oxide thin film layer, and the gate dielectric layer is a lithium-based solid polymer electrolyte thin film layer.

[0007] Preferably, the reconfigurable phototransistor has three operating modes: pure optical operating mode, pure electrical operating mode, and hybrid optoelectronic operating mode.

[0008] Preferably, when light stimulation greater than a first preset energy is applied to the reconfigurable phototransistor, the indium gallium zinc oxide in the channel layer generates electrons through oxygen vacancy ionization. The electrons enter the channel on the channel layer and increase the conductivity of the reconfigurable phototransistor, which exhibits non-volatile characteristics. When the light stimulation is removed, the conductivity gradually decreases over time. When an electrical stimulus greater than a second preset energy is applied to the reconfigurable phototransistor, a double-layer structure is formed at the interface between the gate dielectric layer and the channel layer, and a portion of the Li in the double-layer structure... + It is injected into the channel of the channel layer through electrochemical doping effect and forms a strong interaction with the indium gallium zinc oxide of the channel layer. The reconfigurable phototransistor exhibits non-volatile characteristics; when the electrical stimulation greater than the second preset energy is removed, the conductivity gradually decreases over time.

[0009] Preferably, the gate dielectric layer is formed by filling the channels of the channel layer with a polyethylene oxide-lithium perchlorate composite precursor solution.

[0010] Preferably, the source electrode and the drain electrode are aluminum electrodes.

[0011] Secondly, the present invention proposes a sensor-internal reservoir computing system based on reconfigurable phototransistors, comprising: a preprocessing module and a postprocessing module; The preprocessing module includes an encoding unit, an optical control unit, an electrical control unit, and a reservoir unit. The encoding unit is used to convert the pixel signals of the target image into photoelectric signals. The light control unit is used to apply light stimulation to the storage tank unit based on photoelectric signals; The electrical control unit is used to apply electrical stimulation to the reservoir unit based on photoelectric signals; The reservoir unit is used to generate a reservoir state vector under light stimulation and / or electrical stimulation; The post-processing module is used to classify the target image based on the reservoir state vector; The reservoir unit includes a plurality of reconfigurable phototransistors as described in any one of the first aspects.

[0012] Preferably, the operating modes of the reservoir unit are divided into single-signal mode and multi-signal mode; among which, the single-signal mode is divided into LLLL and EEEE, and the multi-signal mode is divided into LEEE, LLEE and LLLE; where L represents light stimulation and E represents electrical stimulation; both light stimulation and electrical stimulation have two states: on and off.

[0013] Preferably, it also includes a standardization processing module, which is used to standardize the state vector of the reserve pool before it enters the post-processing module.

[0014] Preferably, the post-processing module includes a readout layer unit, which includes a fully connected layer and a ReLU activation function layer connected in sequence. The fully connected layer is used to transform the feature dimension of the reservoir state vector to obtain an intermediate vector; the ReLU activation function layer is used to perform nonlinear mapping processing on the intermediate vector to obtain the classification result of the target image.

[0015] Preferably, a Dropout layer is connected after the fully connected layer.

[0016] Preferably, the number of readout layer units is one or more; when the number of readout layer units is multiple, the multiple readout layer units are divided into a main task level and at least one slave task level, the main task level and at least one slave task level are cascaded, and each slave task level is used to reclassify the classification results of the previous task level.

[0017] This invention presents a sensor-internal reservoir computing system and its reconfigurable phototransistor, which achieves dynamic control of volatile and non-volatile properties under light and / or electrical stimulation in a single device. This solves the technical problem that existing neuromorphic brain-like visual devices struggle to achieve the fusion of volatile and non-volatile characteristics under photoelectric excitation at the single-device level. This design overcomes the functional limitations of existing devices, significantly improving device integration; it also breaks through the von Neumann bottleneck at the architectural level, ultimately greatly improving data transmission and processing efficiency. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the reconfigurable phototransistor in one embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the processing flow of the sensor-internal storage pool calculation system in one embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of the number of pulses and drain-source current in LLEE mode in one embodiment of the present invention.

[0021] Figure 4 This is the reservoir state vector of the reconfigurable phototransistor in LLEE mode in one embodiment of the present invention.

[0022] Figure 5 This is the reservoir state vector of the reconfigurable phototransistor in LLLL mode in one embodiment of the present invention.

[0023] Figure 6 This is the reservoir state vector of the reconfigurable phototransistor in EEEE mode in one embodiment of the present invention.

[0024] Figure 7This is the reservoir state vector of the reconfigurable phototransistor in LEEE mode in one embodiment of the present invention.

[0025] Figure 8 This is the reservoir state vector of the reconfigurable phototransistor in LLLE mode in one embodiment of the present invention.

[0026] Figure 9 This is a schematic diagram of the long-term enhancement and suppression conductivity characteristics of a reconfigurable phototransistor in one embodiment of the present invention.

[0027] Figure 10 This is a schematic diagram of the processing flow of the sensor-internal storage pool calculation system in another embodiment of the present invention.

[0028] Figure 11 This is a graph showing the recognition accuracy of clothing, letters, and numbers in LLEE mode under 50% visibility conditions in one embodiment of the present invention.

[0029] Figure 12 This is a graph showing the final recognition accuracy of multi-task learning under five working modes with different visibility levels in one embodiment of the present invention. Detailed Implementation

[0030] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0031] Firstly, referring to Figure 1 The present invention proposes a reconfigurable phototransistor, comprising: a substrate layer, a channel layer, a gate dielectric layer, and an electrode layer; The channel layer is disposed on the substrate layer; the electrode layer includes a source electrode and a drain electrode, which are disposed on the channel layer with a gap between them; the gate dielectric layer is disposed on the channel layer between the source electrode and the drain electrode, and the gate dielectric layer serves as the gate electrode.

[0032] In this embodiment, the source electrode and the drain electrode are aluminum electrodes.

[0033] This configuration allows the source and drain electrodes to form ohmic contacts with the indium gallium zinc oxide in the channel layer, which is more conducive to the efficient injection and transport of charge carriers.

[0034] In this embodiment, the channel layer is made of indium gallium zinc oxide (IGZO) thin film, and the gate dielectric layer is made of lithium-based solid polymer electrolyte (LSPE) thin film.

[0035] In a further embodiment, the channel layer is made of an indium gallium zinc oxide thin film compatible with silicon CMOS processes, which can meet the needs of neuromorphic sensing and system integration.

[0036] In one specific embodiment, the indium gallium zinc oxide thin film is made of amorphous indium gallium zinc oxide material.

[0037] It is important to understand that amorphous indium gallium zinc oxide (IGaZO) is a promising candidate for channel materials. It has enhanced continuous photoconductivity and good compatibility with silicon CMOS processes, making it suitable for the needs of neuromorphic sensing and system integration. Its tunable optical response is mainly related to the oxygen vacancy concentration.

[0038] In one specific embodiment, the lithium-based solid polymer electrolyte film is made of a lithium-based solid polymer electrolyte.

[0039] It should be understood that lithium-based solid polymer electrolyte materials possess high mechanical stability, a wide electrochemical window, and excellent environmental tolerance. Their tunable electrical properties mainly stem from the double-layer modulation and electrochemical doping effects at the electrolyte / channel interface.

[0040] In one specific embodiment, the gate dielectric layer is formed by filling the channels of the channel layer with a polyethylene oxide (PEO)-lithium perchlorate (LiClO4) composite precursor solution.

[0041] Polyethylene oxide (PEO)-lithium perchlorate (LiClO4) composite precursor solution, as a lithium-based solid polymer electrolyte material, has high mechanical stability, wide electrochemical window and excellent environmental tolerance. Its reconfigurability between volatile and non-volatile states under electrical stimulation is mainly due to the double-layer modulation and electrochemical doping effect at the electrolyte / channel interface.

[0042] It is important to understand the oxygen vacancies (V) in indium gallium zinc oxide (IGZO). o They exist in two forms: shallow donor state and deep localized state. When light stimulation with an energy greater than the first preset energy ionizes oxygen vacancies to generate electrons, the corresponding reaction equation can be expressed as V o →V o + + e - With V o →V o 2+ + 2e - This process allows excess electrons to enter the channel layer, thereby increasing the device's conductivity. With increasing UV light dose, photogenerated electrons accumulate more readily at the interface between the gate dielectric layer and the channel layer, specifically at the interface between the lithium-based solid polymer electrolyte (LSPE) and indium gallium zinc oxide (IGZO), ultimately making the reconfigurable phototransistor non-volatile. When the light stimulus is removed, the Vt in the channel layer... o 2+ With V o+ It will recombine and neutralize with electrons, causing the conductivity to gradually decrease over time.

[0043] When the reconfigurable phototransistor is stimulated with light energy less than or equal to a first preset energy, the reconfigurable phototransistor exhibits typical volatile characteristics.

[0044] In one specific embodiment, the first preset energy is light at 4.70 mW / cm². 2 The energy produced by irradiation for 1 second.

[0045] When an electrical stimulus of less than or equal to a second preset energy is applied to the gate of the reconfigurable phototransistor, the Li in the lithium-based solid polymer electrolyte... + Due to repulsive forces, the electrons are repelled to the interface between the gate dielectric layer and the channel layer, i.e., the interface between the lithium-based solid polymer electrolyte and indium gallium zinc oxide, where they rapidly form an electric double layer (EDL) with the electrons in the channel. After the stimulus is removed, the Li at this interface... + It will spontaneously relax back into the lithium-based solid polymer electrolyte, and the current will eventually recover to the baseline level, exhibiting typical volatile characteristics; When an electrical stimulus greater than a second preset energy is applied to the gate of the reconfigurable phototransistor, the portion of Li enriched in the double layer... + Under the influence of a higher electric field potential energy, it will be injected into the channel in the channel layer through electrochemical doping and form a strong interaction with the indium gallium zinc oxide of the channel layer, thereby enabling the reconfigurable phototransistor to produce stable non-volatile characteristics; wherein, the range of the second preset voltage pulse is greater than the range of the first preset voltage pulse.

[0046] Weak light and / or electrical stimulation makes the reconfigurable phototransistor volatile, while strong light and / or electrical stimulation makes it non-volatile. This embodiment achieves dynamic control of the volatile and non-volatile characteristics of light and electrical stimulation simultaneously on a single device.

[0047] In one specific embodiment, the second preset energy is the energy generated by applying a voltage of 0.6V for 1 second.

[0048] It is important to understand that both light stimulation and electrical stimulation have two states: off and on.

[0049] The reconfigurable phototransistor constructed in this embodiment is based on the dynamic migration of photogenerated oxygen vacancies, the fast response mechanism of the double electric layer, and the electrochemical doping mechanism. It processes the input signal in real time and achieves dynamic control of volatile and non-volatile properties under light and / or electrical stimulation in a single device. This solves the technical problem that existing neuromorphic brain-like visual devices cannot achieve the fusion of volatile and non-volatile characteristics under photoelectric excitation at the single device level. This design breaks through the functional limitations of existing devices and significantly improves the device integration; it also breaks through the von Neumann bottleneck at the architectural level, ultimately greatly improving data transmission and processing efficiency.

[0050] In one specific embodiment, the fabrication process of the reconfigurable phototransistor includes: A silicon dioxide (SiO2) plate is provided as the substrate; A 41.23 nm thick indium gallium zinc oxide (IGZO) thin film was prepared on the surface of a substrate using magnetron sputtering to obtain a channel layer. An 80nm thick aluminum (Al) electrode is deposited on the surface of the channel layer using a thermal evaporation coating process to serve as the source and drain electrodes, forming an ohmic contact with the indium gallium zinc oxide in the channel layer, which is more conducive to the efficient injection and transport of charge carriers. A polyethylene oxide (PEO)-lithium perchlorate (LiClO4) composite precursor solution is filled into the channel using a drop-coating process to form a gate dielectric layer, which is then used as the gate electrode.

[0051] Secondly, referring to Figure 2 The present invention proposes a sensor internal reservoir computing system based on reconfigurable phototransistors, comprising: a preprocessing module and a postprocessing module; The preprocessing module includes an encoding unit, an optical control unit, an electrical control unit, and a reservoir unit. The encoding unit is used to convert the pixel signals of the target image into photoelectric signals. The light control unit is used to apply light stimulation to the storage tank unit based on photoelectric signals; The electrical control unit is used to apply electrical stimulation to the reservoir unit based on photoelectric signals; The reservoir unit is used to generate a reservoir state vector under light stimulation and / or electrical stimulation; The post-processing module is used to classify the target image based on the reservoir state vector; The reservoir unit in this embodiment includes a plurality of reconfigurable phototransistors as described in any one of the first aspects.

[0052] In this embodiment, the reconfigurable phototransistors are arrayed in the reservoir as independent functional nodes. Each reconfigurable phototransistor is a core functional unit of the reservoir, providing high-dimensional state space support for the reservoir by virtue of its inherent photoelectric nonlinearity and volatile dynamic characteristics. This array layout is highly adapted to the dimension of the input information. When processing images, the array nodes can correspond to the spatial distribution of image pixels, converting spatial pixel information into a temporal input sequence.

[0053] The operating modes of the reservoir unit in this embodiment are divided into single-signal mode and multi-signal mode; among them, the single-signal mode is divided into LLLL and EEEE, and the multi-signal mode is divided into LEEE, LLEE and LLLE; where L represents light stimulation and E represents electrical stimulation.

[0054] This embodiment utilizes the volatility of reconfigurable phototransistors under light and electrical stimulation to design a reservoir state comprising two single-signal modes and three multi-signal modes, resulting in a richer and more stable reservoir state. In complex environments, the photoelectrically co-coded reservoir units achieve complementary feature information, enhancing the system's environmental anti-interference capability.

[0055] Furthermore, by switching light stimulation and / or electrical stimulation on and off, the "0" and "1" in a 4-bit binary sequence are simulated respectively, with a total of 16 combinations for each sequence, which are used for the time-series pulse sequence group corresponding to the pixel information of static images.

[0056] In one specific embodiment, in the off state, the parameters of the light stimulation are 0 mW / cm². 2 The electrical stimulation parameters were 0 V, and both light and electrical stimulation lasted for 1 second. In the on state, the light stimulation parameters were 4.70 mW / cm², and the electrical stimulation parameters were 0.6 V, with both light and electrical stimulation lasting for 1 second.

[0057] Therefore, in this embodiment, the operating mode of the reservoir unit can be dynamically controlled by utilizing the volatility generated by light stimulation and / or electrical stimulation.

[0058] It's important to understand that L (light stimulation) being "0" means 1 second of illumination at 340 nm and 0 mW / cm², i.e., in the off state; L (light stimulation) being "1" means 1 second of illumination at 340 nm and 4.70 mW / cm², i.e., in the on state. E (electrical stimulation) being "0" means 1 second of gate voltage stimulation at VGS=0 V, i.e., in the off state; E (electrical stimulation) being "1" means 1 second of illumination at VGS=0.6 V, i.e., in the on state. The sequence "0000-1111" is a 4-bit binary sequence, with the meanings of 0 and 1 as described above.

[0059] In this embodiment, "LLLL, EEEE, LEEE, LLEE, LLLE" correspond to the five operating modes of the storage pool unit, and the meanings of the letters corresponding to the five modes are as follows: Figure 3 As shown in the diagram, LLEE is a 4-bit binary sequence where the first two stimuli are light stimuli and the last two are electrical stimuli; LEEE is a 4-bit binary sequence where the first stimulus is light and the last three are electrical stimuli; LLLE is a 4-bit binary sequence where the first three stimuli are light stimuli and the last stimulus is electrical stimuli; LLLL is a 4-bit binary sequence of four light stimuli; and EEEE is a 4-bit binary sequence of four electrical stimuli.

[0060] Figure 3 This demonstrates the relationship between the number of pulses and the drain-source current in different 4-bit binary sequences under LLEE mode. Figure 3 As can be seen, the current exhibits a stable and distinguishable response with the pulse input, verifying the effective conversion of the photoelectric stimulation timing signal and providing a hardware foundation for the generation of the reservoir state vector.

[0061] Figures 4-8 The state vector distribution of the reservoir under five operating modes is presented. Each mode was tested five times, and the 16 current states under each mode showed significant distinguishability, demonstrating that the multi-signal mode design can enrich the reservoir state space, ensure the diversity of feature extraction, and support classification tasks in complex scenarios.

[0062] Figure 9 This reflects the long-term duration enhancement (LTP) and long-term duration suppression (LTD) conductance characteristics of the reconfigurable phototransistor. The horizontal axis represents the number of pulses, and the vertical axis represents the conductance value. This characteristic provides the physical basis for updating the weight matrix of the fully connected layer in the post-processing module, ensuring the training effectiveness of the classification model.

[0063] Each reconfigurable phototransistor in this embodiment can operate independently in multiple modes, including pure optical (LLLL), pure electrical (EEEE), or hybrid optoelectronic (LLLE, LLEE, and LEEE). By combining "0" (off state) and "1" (on state) of a 4-bit binary sequence, 16 distinguishable current states can be generated, exhibiting high discriminative power and excellent environmental stability, ensuring the reliability and separability of the reservoir output characteristics. Different nodes in the array can respectively receive optical and electrical signal inputs, utilizing optoelectronic complementary characteristics to compensate for information loss under single signal modes and improve feature extraction capabilities in complex scenarios.

[0064] In this embodiment, the output signal of each reconfigurable phototransistor is connected by random synapses to construct a nonlinear dynamic system. In this nonlinear dynamic system, the input photoelectric signal is mapped to a high-dimensional chaotic state space, realizing the extraction of the signal's temporal correlation and nonlinear characteristics, forming a reservoir state vector r(t). This process does not require explicit training of the internal connections of the reservoir units; it can efficiently extract the temporal features (i.e., the reservoir state vector) solely by relying on the nonlinear characteristics of the reconfigurable phototransistor itself.

[0065] The encoding unit in this embodiment includes a photoelectric synaptic device; wherein, converting the pixel signal of the target image into a photoelectric signal specifically includes: Convert the pixel signals of the image into a photoelectric timing pulse sequence; The photoelectric timing pulse sequence is input into the photoelectric synapse device, and the current or conductance signal of the photoelectric synapse device is collected as the photoelectric signal.

[0066] Of course, this embodiment also includes a standardization processing module, which is used to standardize the state vector of the reserve pool before it enters the post-processing module.

[0067] The expression for standardization is: In the formula, Let μ be the state vector of the reservoir that needs to be standardized. Let the standard deviation of the reservoir state vector be denoted as . Indicates the timing step size. This represents the feature vector after standardization.

[0068] In this embodiment, the standardized reservoir state vector is used as the actual input to the post-processing module to ensure stable data distribution to adapt to subsequent network computation, thereby accelerating training convergence.

[0069] In this embodiment, the post-processing module includes a readout layer unit, which includes a fully connected layer and a ReLU activation function layer. The fully connected layer is used to transform the feature dimension of the reservoir state vector to obtain an intermediate vector. The ReLU activation function layer is used to perform nonlinear mapping processing on the intermediate vector to obtain the classification result of the target image, realizing a linear mapping from the high-dimensional feature space to the target output.

[0070] It should be understood that the core processing unit of the post-processing module in this embodiment is a fully connected layer, which achieves feature dimension transformation through a linear transformation formula. The linear transformation formula is: Where W is the weight matrix and b is the bias vector.

[0071] This embodiment also introduces nonlinear mapping capability by connecting a ReLU activation function layer after the fully connected layer, thereby enhancing the post-processing module's ability to express complex features.

[0072] In this embodiment, the weights of the fully connected layer are initialized using the Kaiming normal distribution method to ensure that the initial weight distribution satisfies the desired distribution. `in_features` refers to the dimension of the input features received by the current layer; the bias vector `b` is initialized to 0. This strategy optimizes the characteristics of the ReLU activation function, effectively avoiding the vanishing or exploding gradient problem and improving the stability of the training process. Of course, in this embodiment, the preset weight matrix is ​​obtained by linear regression training using prior external data by the readout layer unit.

[0073] in, Figure 9 This is a schematic diagram of the long-term enhancement and suppression conductance characteristics of the reconfigurable phototransistor, which can be used to update the weight matrix W of the fully connected layer in the readout layer cell.

[0074] In this embodiment, there are one or more readout layer units; when there are multiple readout layer units, the multiple readout layer units are divided into a main task level and at least one slave task level, the main task level and at least one slave task level are cascaded, and each slave task level is used to reclassify the classification results of the previous task level.

[0075] With this setup, this embodiment is suitable for complex recognition and classification environments with multiple cascaded task levels.

[0076] In one embodiment, multiple readout layer units are divided into a main task level and a sub-task level. The main task model (readout layer unit) of the main task level is used to perform preliminary classification on the input target image to obtain the main task classification result. Subsequently, according to the different categories of the main task classification result, the system automatically selects the sub-task model (readout layer unit) in the sub-task level that corresponds to the main task classification result for cascading processing.

[0077] In this embodiment, the readout layer unit in the post-processing module relies on the non-volatile characteristics of the reconfigurable phototransistor in the reservoir unit to achieve stable signal storage and subsequent processing. There is no need to train the internal weights of the reservoir; only the parameters of the readout layer unit need to be optimized, thus adapting to diverse needs such as multi-task image recognition.

[0078] To suppress overfitting, this embodiment also incorporates a Dropout layer after the fully connected layer, which randomly discards neuron outputs with a probability of 0.3 to improve the model's generalization ability.

[0079] In summary, this invention addresses the technical problems of feature loss, poor recognition accuracy, and low computational efficiency in existing single-signal mode reservoir computing systems when recognizing complex static images with incomplete features by setting up a reservoir unit comprising multiple reconfigurable phototransistors. It also solves the technical problems of insufficient richness and stability of the reservoir layer's reserve states and the single response mode in existing reservoir computing systems. Furthermore, by centralizing the reservoir unit and readout layer unit in the same system, it solves the technical problems of low efficiency, high energy consumption, and high hardware loss caused by the separation of sensing, storage, and multi-level processing in traditional von Neumann computing architectures. It also solves the technical problem that traditional computing models (such as recurrent neural networks) are difficult to deploy on edge devices requiring low latency and real-time computing due to error accumulation and cumbersome training. Finally, it addresses the technical problem that existing reservoir computing systems lack task cascading decision-making capabilities, making it difficult to handle complex sequential tasks.

[0080] The present invention will now be described in conjunction with specific embodiments.

[0081] Example 1 This embodiment proposes a sensor-internal reservoir computing system based on reconfigurable phototransistors, including: a preprocessing module, a normalization processing module, and a postprocessing module; The preprocessing module includes an encoding unit, an optical control unit, an electrical control unit, and a reservoir unit. The encoding unit is used to convert the pixel signals of the target image into photoelectric signals. The light control unit is used to apply light stimulation to the storage tank unit based on photoelectric signals; The electrical control unit is used to apply electrical stimulation to the reservoir unit based on photoelectric signals; The reservoir unit is used to generate a reservoir state vector under light stimulation and / or electrical stimulation; The standardization processing module is used to standardize the state vector of the reserve pool; The post-processing module is used to classify the target image based on the standardized reservoir state vector. The post-processing module includes a cascaded main task level and a slave task level. The main task level includes a main task readout layer unit, and the slave task level includes two sub-task readout layer units. Each readout layer unit includes a fully connected layer, a ReLU activation function layer, and a Dropout layer. The reservoir unit includes multiple reconfigurable phototransistors arranged in an array. Each reconfigurable phototransistor includes a substrate layer, a channel layer, a gate dielectric layer, and an electrode layer. The channel layer is disposed on the substrate layer. The electrode layer includes a source electrode and a drain electrode, which are spaced apart on the channel layer. The gate dielectric layer is disposed on the channel layer between the source electrode and the drain electrode, and serves as the gate electrode. The source electrode and the drain electrode are aluminum electrodes, the channel layer is an indium gallium zinc oxide thin film, and the gate dielectric layer is a lithium-based solid polymer electrolyte thin film formed by filling the channel of the channel layer with a polyethylene oxide-lithium perchlorate composite precursor solution.

[0082] The reservoir unit operates in two modes: single-signal mode and multi-signal mode. The single-signal mode is further divided into LLLL and EEEE, while the multi-signal mode is divided into LEEE, LLEE, and LLLE. Here, L represents light stimulation and E represents electrical stimulation. Both light and electrical stimulation have two states: on and off.

[0083] Example 2 This embodiment uses the sensor internal reservoir computing system based on reconfigurable phototransistors described in Embodiment 1 to classify the target image and obtain the classification result of the target image.

[0084] The target images in this embodiment integrate information from the Fashion-MNIST clothing dataset (including locally invisible contamination samples), the E-MNIST alphabet dataset (clothing size information encoding), and the MNIST digit dataset (shoe size information encoding). Each target image in the test underwent binarization and cropping preprocessing (22 pixels wide and 20 pixels long), and was then reassembled into 110 pixels wide and 4 pixels long to enhance local features and improve noise robustness.

[0085] It's important to understand that for contaminated images in the Fashion-MNIST clothing dataset, the feature information of the visible area on the left side of the contaminated image is captured through light pulses, while the invisible area on the right side (containing noise and missing visual information) can only be perceived through touch. The proportion of contaminated pixels in the contaminated image is represented by the proportion of invisible pixels. Pressure sensors typically convert tactile information into electrical signals; in this embodiment, an electrical signal is directly introduced into the device to simulate the tactile information input of a pressure sensor.

[0086] The classification process in this embodiment is as follows: Figure 10As shown. In the specific implementation process, the encoding unit simulates 110 sets of light pulses and / or electrical pulses for the clothing image, so as to convert the pixel signal of the target image into photoelectric signals in LLLL, EEEE, LEEE, LLEE and LLLE modes respectively; among them, LLLL simulates pure visual perception, EEEE simulates pure tactile perception, and LLLE, LLEE and LEEE simulate combined visual and tactile perception; then the photoelectric signals of each mode are input to the reservoir unit for processing through the light control unit and the electrical control unit to obtain the reservoir state vector of the image in each working mode; Then, these reservoir state vectors are first classified into clothing categories by the main task readout layer unit (clothing image classifier) ​​in the post-processing module; once the system recognizes the clothing, one of the subtask readout layer units (letter classifier) ​​in the post-processing module is activated and the corresponding image features are input. Otherwise, if the shoes are identified, another subtask readout layer unit (digit-specific classifier) ​​in the post-processing module is activated to process the corresponding image features, and finally achieve joint recognition of clothing type-letter size / footwear-digit size under pollution conditions.

[0087] Where L represents light stimulation and E represents electrical stimulation, both of which have two states: off and on. In the off state, the parameters of light stimulation are 0 mW / cm², and the parameters of electrical stimulation are 0 V, with both light and electrical stimulation lasting for 1 second. In the on state, the parameters of light stimulation are 4.70 mW / cm², and the parameters of electrical stimulation are 0.6 V, with both light and electrical stimulation lasting for 1 second.

[0088] like Figure 11 As shown, under 50% visibility conditions, the multi-task classification method in LLEE mode achieves independent recognition accuracies of 93.02%, 94.20%, and 99.50% on the Fashion-MNIST clothing dataset, the MNIST digit dataset, and the E-MNIST letter dataset, respectively, with a system-wide recognition accuracy of 91.19%. This target classification method effectively improves the multi-task recognition accuracy in complex and polluted environments through multi-signal reservoirs and task cascade decision-making.

[0089] from Figure 12 It can be observed that under different visibility conditions, the recognition rate of EEEE and LLLL in single-signal mode drops significantly when faced with extremely unfavorable conditions for capturing features; LEEE, LLEE, and LLLE in multi-signal modes all demonstrated good recognition capabilities. This indicates that single-mode approaches have inherent limitations in environments with partially missing information; their feature extraction capabilities rely on complete electrical or optical signal inputs and cannot achieve information complementarity through optoelectronic co-coding like hybrid modes. This verifies their robustness in complex environments. In other words, under various visibility conditions, the combined visual-tactile feature capture capability is the strongest.

[0090] This embodiment demonstrates that employing a multi-signal reservoir unit in the preprocessing module effectively enhances system robustness, and that the post-processing module, through task cascade decision-making, ensures multi-task learning in complex environments, providing crucial experimental evidence for this invention. Compared to traditional methods, this invention significantly reduces trainable parameters while maintaining high-dimensional feature representation capabilities. The preprocessing module's use of a multi-signal reservoir enhances system robustness, and the post-processing module's task cascade decision-making enables multi-task learning, providing an innovative technical path for optoelectronic fusion intelligent systems.

[0091] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A reconfigurable phototransistor, characterized in that, include: Substrate layer, channel layer, gate dielectric layer, and electrode layer; The channel layer is disposed on the substrate layer; the electrode layer includes a source electrode and a drain electrode, which are disposed alternately on the channel layer; the gate dielectric layer is disposed on the channel layer between the source electrode and the drain electrode, and the gate dielectric layer serves as the gate; wherein, the channel layer is an indium gallium zinc oxide thin film layer, and the gate dielectric layer is a lithium-based solid polymer electrolyte thin film layer.

2. The reconfigurable phototransistor according to claim 1, characterized in that, When light stimulation greater than a first preset energy is applied to the reconfigurable phototransistor, the indium gallium zinc oxide in the channel layer generates electrons through oxygen vacancy ionization. The electrons enter the channel on the channel layer and increase the conductance of the reconfigurable phototransistor, which exhibits non-volatile characteristics. When the light stimulation is removed, the conductance gradually decreases over time. When an electrical stimulus greater than a second preset energy is applied to the reconfigurable phototransistor, a double-layer structure is formed at the interface between the gate dielectric layer and the channel layer, and a portion of the Li in the double-layer structure... + It is injected into the channel of the channel layer through electrochemical doping effect and forms a strong interaction with the indium gallium zinc oxide of the channel layer. The reconfigurable phototransistor exhibits non-volatile characteristics; when the electrical stimulation is removed, the conductivity gradually decreases over time.

3. The reconfigurable phototransistor according to claim 1, characterized in that, The reconfigurable phototransistor has three operating modes: pure optical mode, pure electrical mode, and hybrid optoelectronic mode.

4. The reconfigurable phototransistor according to claim 1, characterized in that, The gate dielectric layer is formed by filling the channels of the channel layer with a polyethylene oxide-lithium perchlorate composite precursor solution.

5. A sensor-internal reservoir computing system based on reconfigurable phototransistors, comprising: Preprocessing module and postprocessing module; The preprocessing module includes an encoding unit, an optical control unit, an electrical control unit, and a reservoir unit. The encoding unit is used to convert the pixel signals of the target image into photoelectric signals. The light control unit is used to apply light stimulation to the storage tank unit based on photoelectric signals; The electrical control unit is used to apply electrical stimulation to the reservoir unit based on photoelectric signals; The reservoir unit is used to generate a reservoir state vector under light and / or electrical stimulation; The post-processing module is used to classify the target image based on the reservoir state vector; The reservoir unit includes a plurality of reconfigurable phototransistors as described in any one of claims 1-4.

6. The sensor-internal reservoir computing system based on reconfigurable phototransistors according to claim 5, characterized in that, The reservoir unit operates in two modes: single-signal mode and multi-signal mode. The single-signal mode is further divided into LLLL and EEEE, while the multi-signal mode is divided into LEEE, LLEE, and LLLE. Here, L represents light stimulation and E represents electrical stimulation. Both light and electrical stimulation have two states: on and off.

7. The sensor-internal reservoir computing system based on reconfigurable phototransistors according to claim 5, characterized in that, It also includes a standardization processing module, which is used to standardize the state vector of the reserve pool before it enters the post-processing module.

8. The sensor-internal reservoir computing system based on reconfigurable phototransistors according to claim 5, characterized in that, The post-processing module includes a readout layer unit, which consists of a fully connected layer and a ReLU activation function layer connected in sequence. The fully connected layer is used to transform the feature dimension of the reservoir state vector to obtain an intermediate vector; the ReLU activation function layer is used to perform non-linear mapping processing on the intermediate vector to obtain the classification result of the target image.

9. The sensor-internal reservoir computing system based on reconfigurable phototransistors according to claim 8, characterized in that, A Dropout layer is connected after the fully connected layer.

10. The sensor-internal reservoir computing system based on reconfigurable phototransistors according to claim 8, characterized in that, The number of readout layer units can be one or more; When there are multiple readout layer units, the multiple readout layer units are divided into a master task level and at least one slave task level. The master task level and at least one slave task level are cascaded, and each slave task level is used to reclassify the classification results of the previous task level.

Images