An artificial visual nervous system based on dual-mode neural devices
By designing an artificial visual neural system based on bimodal neural devices, and using physical convolutional kernel arrays and synaptic computing arrays to simulate the biological retina and central nervous system, low-power and high-efficiency image processing and recognition are achieved, solving the problems of low computational efficiency and high energy consumption in existing technologies.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIAN JIAOTONG LIVERPOOL UNIV
- Filing Date
- 2025-04-29
- Publication Date
- 2026-05-22
Smart Images
Figure CN120471114B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of semiconductor technology, and in particular to an artificial visual neural system based on a bimodal neural device. Background Technology
[0002] With the rise of artificial intelligence and artificial neural networks, machine vision has been widely used in various fields such as autonomous driving, robot navigation, medical image processing, and security monitoring. The core objective of machine vision is to collect visual data of the surrounding environment through cameras or other sensors, and then analyze and process this data to complete specific tasks. Convolutional operations are frequently used in the data analysis and processing stage to extract different features from images. However, such systems based on traditional hardware often suffer from low information processing efficiency and high energy consumption due to the separation of various sensing, analog-to-digital conversion, computing, and storage modules. Inspired by biological nervous systems, researchers have recently begun to physically realize biological visual nervous systems. Biological visual nervous systems are divided into peripheral visual nervous systems and central visual nervous systems.
[0003] The peripheral visual nervous system is primarily responsible for visual signal perception and feature extraction (information preprocessing), with the retina being its main component. These two main functions of the retina are performed by rod and cone cells in the upper layer and bipolar cells in the lower layer, respectively. In the photoreceptor layer, rod cells are primarily responsible for visual perception in low-light environments, sensing light intensity but not color recognition. Cone cells are primarily responsible for color vision and detail recognition in bright environments. There are three types of cone cells, sensitive to red, green, and blue light respectively. Bipolar cells, located between photoreceptor cells and ganglion cells, are responsible for transmitting light signals from photoreceptor cells to ganglion cells and, to some extent, performing convolution-like processing and integration of the signals. The central visual nervous system, located in the brain, integrates and recognizes the preprocessed feature information. This function mainly relies on the biological synaptic structures within complex neural networks. Researchers hope to build a novel neuromorphic vision system by fabricating electronic units with neural-simulated behavior, thereby achieving a low-power, highly integrated miniaturized image sensing and computing architecture.
[0004] Currently, some teams have fabricated artificial retinas for light perception and artificial synapses for neural network computation. However, most current artificial retinas can only mimic the photoreceptor cells in the biological retina, possessing only the ability to generate basic non-volatile photocurrents and integrate low-level signals, but cannot simultaneously generate positive and negative photocurrents, thus making it difficult to achieve convolution operations. Furthermore, devices capable of generating optical signal convolution operations can only process optical signals, making it difficult to integrate multimodal biomimetic neural functions into a single component.
[0005] Therefore, designing a novel visual convolution system that can simultaneously perform optical signal convolution operations and salient computation using only a single component is a technical problem that urgently needs to be solved. Summary of the Invention
[0006] The purpose of this invention is to solve the problem in the prior art that multiple types of semiconductor devices are needed to simulate the biological visual nervous system.
[0007] This application provides an artificial visual neural system based on a bimodal neural device, including a bimodal visual information computing array composed of a physical convolutional kernel array and a synaptic computing array. The physical convolutional kernel array is used to receive ambient light signals and convert them into electrical signals for output. The synaptic computing array is used to receive the electrical signals and convert them into visual signals. Both the physical convolutional kernel array and the synaptic computing array are composed of several bimodal transistors. The bimodal transistors have the ability to sense light signals and perform synaptic computing.
[0008] As a further improvement of this application, the dual-mode transistor includes, from bottom to top, a substrate, a dielectric layer, a semiconductor layer, a source and a drain respectively disposed on both sides of the semiconductor layer away from the surface of the dielectric layer, and a light-absorbing layer disposed on the semiconductor layer away from the surface of the dielectric layer and located between the source and the drain.
[0009] As a further improvement of this application, the substrate is a P-type silicon substrate.
[0010] As a further improvement to this application, the dielectric layer material is selected from Al2O3 and ZrO. x HfO x Any one of them.
[0011] As a further improvement to this application, the semiconductor layer is made of an n-type semiconductor material.
[0012] As a further improvement to this application, the semiconductor layer material is selected from InO. x Any one of IGZO, IZO, or IGZO.
[0013] As a further improvement of this application, the light-absorbing layer material is selected from any one of PbS, perovskite, MXene, and MoFs.
[0014] As a further improvement to this application, the fabrication of the dual-mode transistor includes the following steps:
[0015] S1. Clean the substrate;
[0016] S2. A dielectric layer is grown on the upper surface of the substrate;
[0017] S3. A semiconductor layer is grown on the surface of the dielectric layer away from the substrate;
[0018] S4. Prepare source and drain electrodes on the surface of the semiconductor layer away from the dielectric layer;
[0019] S5. A light-absorbing layer is grown on the surface of the semiconductor layer away from the dielectric layer and between the source electrode and the drain electrode.
[0020] As a further improvement of this application, the dual-mode transistor is prepared by solution method to fabricate the dielectric layer, semiconductor layer and light absorption layer.
[0021] As a further improvement of this application, a substrate hydrophilic treatment step is also included between step S1 and step S2.
[0022] The beneficial effects of this application are:
[0023] 1. The dual-mode transistor designed in this application possesses optoelectronic fusion characteristics and non-volatile storage characteristics, and has optical signal sensing capabilities. Through the synergistic design of an optical absorption layer (such as PbS quantum dots) and an n-type semiconductor layer (InOx, etc.), efficient optical signal-to-electrical signal conversion and gain adjustment are achieved. This is achieved through AlO... x The ion migration effect of the dielectric layer endows the device with long-term enhancement / suppression characteristics, supporting online adjustment of synaptic weights.
[0024] 2. The physical convolutional kernel array constructed in this application can simulate the collaborative visual signal processing behavior of photoreceptor cells and bipolar cells in the retina, possessing the functions of visual information detection, processing, and recognition. Through different gate voltage adjustment strategies, various dynamic configurations of the convolutional kernel array (such as sharpening, edge detection, blurring, etc.) can be obtained, providing flexible image processing capabilities. This application can be used to implement visual information processing and recognition operations, and can be applied to neuromorphic image feature enhancement systems in future machine vision.
[0025] 3. This application has excellent dual-modal collaborative computing capabilities. It simulates the receptive field mechanism (excitation / inhibition response) of retinal bipolar cells through a physical convolution kernel array to achieve image feature extraction and preprocessing. Combined with the synaptic plasticity (LTP / LTD mechanism) of the bionic central nervous system of the synaptic computing array, it completes advanced computing of visual signals and forms a complete optic nerve bionic closed loop with the help of a single device. Attached Figure Description
[0026] Figure 1 This is a schematic diagram of the visual convolution system in this application;
[0027] Figure 2 This is a schematic diagram of the dual-mode transistor in this application;
[0028] Figure 3This is a schematic diagram of the structure of a biological retina;
[0029] Figure 4 This is a schematic diagram illustrating the differences in photosensitivity between different types of bipolar cells in this application;
[0030] Figure 5 It is the excitatory photocurrent of the dual-mode transistor in this application;
[0031] Figure 6 This is the suppressive photocurrent of the dual-mode transistor in this application;
[0032] Figure 7 This refers to the change in the current response intensity of the dual-mode transistor in this application;
[0033] Figure 8 This is the equivalent circuit diagram of the physical convolution kernel matrix in this application;
[0034] Figure 9 These are different gate voltage adjustment strategies for the convolution kernel matrix in this application;
[0035] Figure 10 This is a schematic diagram of image processing using the convolution kernel matrix in this application;
[0036] Figure 11 This is the ID-VG scan curve of the dual-mode transistor in this application;
[0037] Figure 12 This is the non-volatile current profile of the dual-mode transistor in this application;
[0038] Figure 13 These are the LTP and LTD conductance values of the dual-mode transistor in this application. Detailed Implementation
[0039] To make the objectives, technical solutions, and advantages of this application clearer, the specific embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0040] In the following description of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.
[0041] In the following description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. Furthermore, the technical features involved in the different embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.
[0042] like Figure 1 As shown, this application provides a visual convolution system based on bimodal neural devices. A bimodal visual information computation matrix is constructed using bimodal neural devices (bimodal transistors) with photoelectric response capabilities. The computation matrix includes a physical convolution kernel array and a synaptic computation array. The physical convolution kernel array processes the input ambient light signal, extracts image feature information, and ultimately outputs an electrical signal. The synaptic computation array receives the electrical signal and ultimately converts it into a visual electrical signal for output. The visual convolution system designed in this application can simulate the peripheral visual nervous system (visual information receptive layer) and the central visual nervous system (visual information computation layer) in the biological visual nervous system.
[0043] like Figure 2 As shown, the dual-mode transistor, from bottom to top, includes a substrate (gate), a dielectric layer, a semiconductor layer, a source and a drain electrode respectively disposed on opposite sides of the semiconductor layer away from the surface of the dielectric layer, and a light-absorbing layer disposed on the semiconductor layer away from the surface of the dielectric layer and located between the source and drain electrodes. The substrate is a heavily doped Si(p++) substrate; the dielectric layer material is a metal oxide semiconductor material with a large bandgap, such as Al2O3 or ZrO. x HfO x Any one of the following; the semiconductor layer is selected from n-type metal-oxide-semiconductor or n-type van der Waals semiconductor materials, such as InO. xThe light-absorbing layer material can be any one of IGZO, IZO, or other light-absorbing materials. It can also be a photosensitive material with effective absorption in the visible light band, such as any one of PbS, perovskite, MXene, or metal-organic frameworks (MOFs).
[0044] This application also provides a method for fabricating a dual-mode transistor. Taking a substrate (gate) of highly doped Si (p++), a dielectric layer of AlOx, a semiconductor layer of InOx, a light-absorbing layer of Pbs, and source / drain electrodes of aluminum as an example, the specific fabrication method is as follows:
[0045] S1: Substrate cleaning: The heavily doped Si(p++) substrate was ultrasonically cleaned with deionized water for 15 min, and then dried under N2 gas flow. Hydrophilic treatment: The cleaned substrate was further treated with plasma for 15 min to make the film surface hydrophilic.
[0046] S2: Prepare the dielectric layer by spin-coating Al2O3-Li+ precursor solution onto the substrate at 3000–5000 rpm to prepare AlO2. x Spin coat the film for 15–30 seconds, then anneal it in air or nitrogen at 300–450°C for 30–60 minutes. During the preparation process, select an appropriate spin coating speed and time based on the preset dielectric layer thickness, and gradually increase the temperature during dielectric layer annealing to avoid cracking of the dielectric film.
[0047] S3: Fabrication of semiconductor layer, AlO2 to be prepared x After the film is cooled, InO x The precursor solution is spin-coated onto the dielectric layer surface at 2000–4000 rpm for 20–40 s, and then annealed at 300–500 °C for 40–100 min. In the specific preparation process, the appropriate spin-coating speed and time are selected according to the preset semiconductor layer thickness, and the temperature should be gradually increased during the semiconductor layer annealing process to avoid cracking of the semiconductor layer film.
[0048] S4: Source and drain electrodes were fabricated. A 30nm thick aluminum metal source / drain electrode was fabricated using a thermal evaporation method.
[0049] S5: Prepare the light absorption layer. Spin-coat the prepared PbS quantum dot dispersion onto the semiconductor layer at 1000-2000 rpm for 20-40 s, and then anneal at 100-300℃ for 30 min to obtain a dual-mode transistor.
[0050] The biomimetic principle of the peripheral optic nerve in this application is as follows:
[0051] The peripheral visual nervous system is primarily responsible for the perception and feature extraction (information preprocessing) of visual signals, and its main component is the retina. For example... Figure 3As shown, the retina is a key component of the peripheral visual nervous system, responsible for sensing and processing external light signals, converting them into neural signals, and then transmitting them to the brain for further analysis and understanding. The retina's function extends beyond simply receiving light; it also performs preliminary image processing tasks, including image enhancement, contrast adjustment, and motion detection. In this process, the retina plays a complex perceptual and processing role, ensuring that the brain receives efficient and useful visual information. The photoreceptor cells of the retina (rod cells and cone cells) are the primary photoreceptors. Rod cells are sensitive in low-light environments and can perceive light intensity, while cone cells can perceive color and are responsible for capturing details. Photoreceptor cells convert light into electrical signals through photosensitive substances (such as rhodopsin).
[0052] Bipolar cells are located in the middle layer of the retina, between photoreceptor cells and ganglion cells. Based on their signal transmission mechanisms, bipolar cells are mainly divided into two types: excitatory bipolar cells (ON bipolar cells): These bipolar cells are primarily associated with the ON (brightness) response of photoreceptor cells. They respond positively to increased light. When photoreceptor cells sense increased light intensity, excitatory bipolar cells depolarize (potential rise), transmitting the signal to downstream ganglion cells. Inhibitory bipolar cells (OFF bipolar cells): These bipolar cells are associated with the OFF (darkness) response of photoreceptor cells. They are activated when light intensity decreases, producing a depolarization response and transmitting the signal to downstream ganglion cells. Simply put, the stronger the light stimulus, the more excited the excitatory bipolar cells become, and conversely, the more inhibited the inhibitory bipolar cells are inhibited.
[0053] The ability to generate two opposing cellular responses to the same light stimulus is a key feature of the biological visual system, contributing significantly to its superior coding performance. This feature is particularly crucial in image enhancement tasks. The spatial region comprised of all stimulus sites capable of triggering a cellular response is called the cell's receptive field. Figure 4 As shown, the receptive field of a bipolar cell is generally a concentric circular structure, consisting of a central circle and an outer ring. The former is called the receptive field center, and the latter is called the receptive field periphery. Based on the type of bipolar cells in the center and periphery, they can be divided into two types: light-exposed and light-removed. They produce different, even opposite, responses to the same stimulus applied to the center and periphery of the receptive field. For example, some bipolar cells show increased activation in the center of the receptive field when brightness increases, but decreased activation in the periphery when brightness increases. Other bipolar cells show increased activation in the center when brightness decreases, but decreased activation in the periphery when brightness decreases. Thus, the receptive field formed by bipolar cells completes the encoding and integration of visual signals in a localized area.
[0054] This application employs a dual-mode transistor to construct a physical convolution kernel array, utilizing convolutional computation to simulate the different responses of bipolar cells to light stimulation. The dual-mode transistor designed in this application can use an optical signal as input, and by changing the applied gate voltage, excitability (e.g., ...) can be achieved. Figure 5 (as shown) and inhibition (such as) Figure 6 (As shown) Two types of photocurrents. When a positive gate voltage is applied, the uppermost photoabsorbing layer generates photogenerated carriers after receiving light stimulation. Electrons drift downwards under the influence of the electric field and enter the semiconductor layer. Since electrons are the main carriers in the n-type semiconductor, the carrier concentration increases after receiving electrons from the photoabsorbing layer, resulting in current gain and exhibiting excitability. When a negative gate voltage is applied, the photoabsorbing layer generates photogenerated carriers after receiving light stimulation. Electrons drift upwards under the influence of the electric field, leaving holes to accumulate at the interface between the photoabsorbing layer and the semiconductor layer. At this time, electrons in the semiconductor layer also drift upwards under the longitudinal electric field and recombine with holes at the interface. This process reduces the electron concentration in the semiconductor layer, decreases the current, and exhibits inhibition. Figure 7 As shown, this application can also change the intensity of the current response by changing the magnitude of the forward gate voltage; the larger the forward gate voltage, the greater the intensity of the current response.
[0055] In some specific embodiments of this application, a dual-mode transistor is used, which is prepared by the following method:
[0056] The substrate was cleaned by ultrasonic cleaning with deionized water for 15 min, followed by drying under N2 gas flow. For hydrophilic treatment, the cleaned substrate was further treated with plasma for 15 min to hydrophilize the film surface. An AlOx thin film was prepared by spin-coating an Al2O3-Li+ precursor solution onto the treated substrate at 4500 rpm for 20 s, followed by annealing at 320 °C in air for 60 min. After the AlOx film cooled, an InOx precursor solution was spin-coated onto the dielectric layer surface at 3500 rpm for 30 s, followed by annealing at 270 °C for 60 min. A 30 nm thick aluminum metal source / drain electrode was prepared on the semiconductor layer surface using thermal evaporation. The prepared PbS quantum dot dispersion was then spin-coated onto the semiconductor layer at 2000 rpm for 20 s, followed by annealing at 200 °C for 30 min to obtain a dual-mode transistor.
[0057] By wire-connecting and packaging the nine fabricated dual-mode transistors, a 3×3 physical convolution kernel array can be obtained, the equivalent circuit diagram of which is shown below. Figure 8 As shown. By applying different gate voltage strategies to each transistor, several common types of convolutional kernels used for image enhancement and feature extraction can be implemented. For example... Figure 9 As shown, for T 1,1 When a positive gate voltage is applied to the cell, T 0,1 T1,0 T 1,2 T 2,1 Applying a negative gate voltage yields a convolution kernel for image sharpening. For T... 1,1 Applying a positive gate voltage to one unit and a negative gate voltage to the remaining eight units yields a convolutional kernel for image edge detection. Applying a small positive gate voltage to each unit yields a convolutional kernel for image blurring. For example... Figure 10 As shown, the physical convolution kernel is used to perform sharpening, edge detection, and image blurring simulation processing on frog images. The physical convolution kernel array designed in this application demonstrates excellent image feature extraction capabilities.
[0058] The biomimetic principle of the central optic nerve in this application is as follows:
[0059] Biological synapses are the core units of information and computation in the central nervous system. Through synaptic plasticity, they dynamically regulate the strength of connections between neurons, forming the physiological basis of learning and memory. Long-term potentiation (LTP) and long-term inhibition (LTD) are key mechanisms of non-volatile weight regulation: high-frequency neural activity triggers LTP, promoting an increase in the number of postsynaptic membrane receptors through calcium ion influx, thus persistently enhancing signal transmission efficiency; low-frequency activity, on the other hand, reduces receptor density through LTD, weakening connection strength over the long term. This bidirectional regulation of synaptic weights based on activity patterns enables biological neural networks to adapt to their environment.
[0060] Artificial synapses, drawing inspiration from biological mechanisms, should possess the ability to simulate non-volatile weighting. By controlling the conductance (weights) of devices through the amplitude, frequency, or timing of electrical pulses, they can achieve "write-hold-erase" characteristics equivalent to those of biological synapses. This biomimetic design enables neuromorphic chips to achieve parallel computing and online learning with extremely low power consumption, providing a hardware foundation for overcoming the bottlenecks of the von Neumann architecture.
[0061] The dual-mode transistor designed in this application has excellent synaptic computing capabilities. For example... Figure 11 As shown, the channel current exhibits significant hysteresis when the device is subjected to a bidirectional gate voltage scan. The observed memory window of approximately 2V is key to the device exhibiting non-volatile current. When a positive voltage pulse is applied to the gate, lithium ions will flow from AlO₂... x Migration to the channel leads to improved channel currents and the formation of excitatory postsynaptic currents. For example... Figure 12 As shown, when the pulse stimulation ends, ions near the channel electrolyte interface will be released due to Li + As the concentration gradient diffuses away from the interface, the current decreases slowly, exhibiting long-term non-volatility, corresponding to long-term enhancement in biological synapses. Subsequently, when the presynaptic stimulus is negative, the downward trend of the channel helps to inhibit the postsynaptic current, thus generating a long-term inhibitory current. Figure 13As shown, normalizing the LTP (Long-Term Potentiation) and LTD (Long-Term Depression) conductivities of the device can be defined as the weight modulation rule in the artificial neural network, which participates in the weight deployment in the subsequent array network.
[0062] The embodiments of the present invention have been described in detail above, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.
Claims
1. An artificial visual neural system based on a bimodal neural device, characterized in that, The system includes a dual-modal visual information computing array composed of a physical convolutional kernel array and a synaptic computing array. The physical convolutional kernel array receives ambient light signals and converts them into electrical signals for output. The synaptic computing array receives the electrical signals and converts them into visual signals. Both the physical convolutional kernel array and the synaptic computing array are composed of several dual-modal transistors, which have optical signal sensing and synaptic computing capabilities. From bottom to top, the dual-modal transistor includes a substrate, a dielectric layer, a semiconductor layer, source and drain electrodes respectively disposed on opposite sides of the semiconductor layer away from the dielectric layer surface, and a light-absorbing layer disposed on the semiconductor layer away from the dielectric layer surface and located between the source and drain electrodes. The dielectric layer material is selected from Al2O3 and ZrO. x HfO x Any one of the following; the semiconductor layer is selected from n-type semiconductor material InO. x The optical absorption layer material is selected from any one of IGZO and IZO; the optical absorption layer material is selected from any one of PbS, perovskite, MXene, and MoFs; the dual-mode transistor takes the optical signal as input and realizes both excitatory and inhibitory photocurrents by changing the applied gate voltage; by applying different gate voltage strategies to each transistor, convolution kernels for image enhancement and feature extraction are realized.
2. The artificial visual nervous system according to claim 1, characterized in that, The substrate is a P-type silicon substrate.
3. The artificial visual nervous system according to claim 1, characterized in that, The fabrication of the dual-mode transistor includes the following steps: S1. Clean the substrate; S2. A dielectric layer is grown on the upper surface of the substrate; S3. A semiconductor layer is grown on the surface of the dielectric layer away from the substrate; S4. Prepare source and drain electrodes on the surface of the semiconductor layer away from the dielectric layer; S5. A light-absorbing layer is grown or transferred on the surface of the semiconductor layer away from the dielectric layer and between the source electrode and the drain electrode.
4. The artificial visual nervous system according to claim 3, characterized in that, The dual-mode transistor is fabricated using a solution method to prepare the dielectric layer, semiconductor layer, and light-absorbing layer.
5. The artificial visual nervous system according to claim 4, characterized in that, The step between steps S1 and S2 also includes a substrate hydrophilic treatment step.