A multi-modal artificial neuron circuit, spiking neural network and electronic device

Abstract

This application discloses a multimodal artificial neuron circuit, a spiking neural network, and an electronic device, relating to the field of integrated circuit technology. Specifically, it includes a comparator module, a pulse generation module, a dynamic voltage path formed by a memristor element and a first impedance element connected in series, and a dynamic threshold path formed by a first sensing element and a second impedance element connected in series. The memristor element is connected to the first input terminal of the comparator module, converting the first modal signal into an accumulated dynamic voltage; the first sensing element is connected to the second input terminal of the comparator module, converting the second modal signal into a real-time dynamic threshold; and the pulse generation module outputs a pulse signal when the dynamic voltage reaches the dynamic threshold. This application utilizes the resistance switching characteristics of the memristor element to simulate the leakage integral change of the neuron's membrane potential, while simultaneously using the first sensing element to convert the environmental modal signal into a dynamic threshold, solving the problems of complexity, low integration, and lack of multimodal fusion in traditional CMOS neuron circuits.

Description

Technical Field

[0001] This application relates to the field of integrated circuit technology, and in particular to a multimodal artificial neuron circuit, a spiking neural network, and an electronic device. Background Technology

[0002] Leaky Integrate-and-Fire (LIF) artificial neural network circuits are mainly used to build spiking neural networks. In scenarios such as intelligent driving and edge intelligence, they are mainly used to encode continuous analog signals collected by underlying sensors into discrete pulse sequences to support event-driven low-power computing paradigms and provide underlying hardware support for environmental perception and automatic decision-making in upper-level systems.

[0003] In existing technologies, a peripheral circuit scheme based on pure CMOS (Complementary Metal-Oxide-Semiconductor) technology is typically used to implement the leak-integrating and firing neuron. Specifically, the traditional implementation mainly relies on combining a large number of conventional transistors to build discrete circuit modules such as operational amplifiers, comparators, reset networks, and static capacitors to construct a complete analog integration and firing circuit, thereby completing the numerical accumulation and level triggering of the input signal.

[0004] However, the aforementioned prior art has the following shortcomings: the prior art uses a large number of conventional discrete transistors to build peripheral analog circuits, while the CMOS pulse coding implementation requires a digital-to-analog converter with high hardware overhead and complex control circuits, resulting in high hardware overhead and high circuit complexity for a single artificial neuron, which limits the large-scale integration of neuron arrays on a single chip.

[0005] Therefore, this application aims to solve the problem of limited integration caused by the complexity of the peripheral analog structure of traditional neuron circuits. Summary of the Invention

[0006] The main objective of this application is to provide a multimodal artificial neuron circuit, a spiking neural network, and an electronic device, aiming to solve the problem of limited integration caused by the complex peripheral analog structure of traditional neuron circuits.

[0007] To achieve the above objectives, this application proposes a multimodal artificial neuron circuit, comprising: Comparison module; A pulse generation module is connected to the output terminal of the comparison module; A memristor and a first impedance element are connected in series, and their common terminal is connected to the first input terminal of the comparison module. The memristor is used to receive a first mode signal and output a dynamic voltage to the first input terminal. A first sensing element and a second impedance element are connected in series, and their common terminal is connected to the second input terminal of the comparison module. The first sensing element is used to receive the second modal signal and output a dynamic threshold to the second input terminal. The pulse generation module outputs a pulse signal when the dynamic voltage reaches the dynamic threshold.

[0008] Furthermore, the pulse generation module also includes a timing element; The pulse generation module has a discharge terminal and a trigger terminal. The trigger terminal is connected to the output terminal of the comparison module, and the discharge terminal is connected to the timing element.

[0009] Furthermore, the timing element includes a second sensing element and an energy storage element, the second sensing element and the energy storage element are connected in series, and the common terminal of the second sensing element and the energy storage element is connected to the discharge terminal of the pulse generation module; The second sensing element is used to receive the second modal signal and change its own resistance according to the second modal signal to adjust the width of the pulse signal.

[0010] Furthermore, the energy storage element is an integrating capacitor; The capacitance value of the integrating capacitor is configured to be negatively correlated with the rate of change of the first modal signal into the dynamic voltage.

[0011] Furthermore, both the first sensing element and the second sensing element include a photoresistor.

[0012] Furthermore, both the first sensing element and the second sensing element are configured such that their resistance values ​​are negatively correlated with the intensity of the second mode signal; The resistance of the first sensing element decreases as the second modal signal increases, thereby reducing the dynamic threshold. The resistance of the second sensing element decreases as the second modal signal increases, thereby reducing the width of the pulse signal.

[0013] Furthermore, the non-common terminal of the first impedance element and the non-common terminal of the first sensing element are both connected to the ground terminal, and the non-common terminal of the second impedance element is connected to the power supply terminal; The inverting input terminal of the comparison module serves as the first input terminal to monitor the dynamic voltage; The in-phase input of the comparison module serves as the second input to obtain the dynamic threshold.

[0014] Furthermore, it also includes pull-up resistors; The output terminal of the comparison module is connected to the power supply terminal via the pull-up resistor, and the output terminal of the comparison module is the low-level trigger node of the pulse generation module.

[0015] This application also discloses a spiking neural network, including: A multi-mode signal input interface is configured to receive the first mode signal and the second mode signal in parallel. The neuron array contains multiple multimodal artificial neuron circuits as described above. The multiple multimodal artificial neuron circuits receive and fuse the first modal signal and the second modal signal through the multimodal signal input interface.

[0016] This application also discloses an electronic device including the aforementioned spiking neural network.

[0017] The above technical solution has the following advantages: This application connects a dynamic voltage path composed of a memristor and a first impedance element to the first input terminal of the comparison module, and a dynamic threshold path composed of a first sensing element and a second impedance element to the second input terminal. This allows the memristor to simulate the leakage integral change of the neuron membrane potential using its resistance switching characteristics, while the first sensing element converts the environmental modal signal into a dynamic threshold. This effectively solves the problems of large hardware overhead, limited integration, and inability to fuse multimodal information caused by the stacking of a large number of transistors in traditional CMOS neuron circuits. It achieves the fusion and real-time sensing of different modal signals under a minimal circuit structure. Attached Figure Description

[0018] The present application will now be described in detail with reference to specific embodiments and accompanying drawings, wherein: Figure 1 This is a structural block diagram of this application; Figure 2 This is the circuit schematic diagram of this application; Figure 3 This is a basic electrical performance diagram of the memristor element in this application; Figure 4 This is a diagram showing the repeated current of the memristor element in this application under multiple sets of the same voltage. Figure 5 This is a scatter plot of the memristor current versus pulse width in this application; Figure 6 This is a scatter plot of the memristor current versus pulse amplitude in this application; Figure 7 This is a timing diagram of the waveforms of each node of the LIF neuron in this application; Figure 8 This is a graph showing the width of the output pulse signal as a function of resistance in this application. Figure 9This is a current-time curve of the memristor element in this application under different equivalent drive voltages.

[0019] In the figure: 10, first sensing element; 20, second impedance element; 30, memristor element; 40, first impedance element; 50, comparison module; 60, pulse generation module; 70, timing element; 71, second sensing element; 72, energy storage element; R1, pull-up resistor. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the following specific embodiments are merely illustrative of this application and do not constitute a limitation thereof.

[0021] Existing leak-integration-electrode (LIF) neuron circuits employ a peripheral circuit scheme based on pure CMOS technology to implement the leak-integration-electrode neuron. This mainly relies on combining a large number of conventional transistors to build discrete circuit modules such as operational amplifiers, comparators, reset networks, and static capacitors to construct a complete analog integration and emission circuit, thereby completing the numerical accumulation and level triggering of the input signal. Conventional discrete transistors are used to build peripheral analog circuits, while the pulse coding implementation of CMOS requires high hardware overhead digital-to-analog converters and complex control circuits, resulting in high hardware overhead and high circuit complexity for a single artificial neuron, which limits the large-scale integration of neuron arrays on a single chip.

[0022] Furthermore, existing technologies, in order to address the above problems, employ fixed-structure peripheral analog circuits, resulting in weak information fusion capabilities when dealing with complex multi-source sensing scenarios. The core parameters of existing neuronal circuits, such as the pulse trigger threshold and pulse width, rely on fixed matching resistors or adjustable resistors that require manual adjustment. This prevents the establishment of adaptive feedback paths to multimodal environmental signals, causing neurons to be unable to dynamically adjust their excitation characteristics and recovery cycles in real time based on external multimodal environmental signals during operation.

[0023] Based on the analysis of the prior art, this application aims to solve the problems of high hardware overhead and high circuit complexity of a single artificial neuron, which limits the large-scale integration of neuron arrays on a single chip. It also solves the defects of hardware-level multimodal fusion and static parameter fixation in the prior art by adopting a neuron circuit based on dynamic feedback adjustment of multimodal environmental signals.

[0024] The following are one or more implementation methods listed in this application to address the above-mentioned technical problems, in order to better solve the above-mentioned technical problems. These include various embodiments, and the specific embodiment schemes are as follows: like Figure 1 and Figure 2 As shown, this embodiment discloses a multimodal artificial neuron circuit, including a comparison module 50, a pulse generation module 60, a memristor element 30, a first impedance element 40, a first sensing element 10, and a second impedance element 20. The pulse generation module 60 is connected to the output terminal of the comparison module 50; the memristor element 30 and the first impedance element 40 are connected in series, and their common terminal is connected to the first input terminal of the comparison module 50. The memristor element 30 is used to receive a first modal signal and output a dynamic voltage to the first input terminal; the first sensing element 10 and the second impedance element 20 are connected in series, and their common terminal is connected to the second input terminal of the comparison module 50. The first sensing element 10 is used to receive a second modal signal and output a dynamic threshold to the second input terminal. The pulse generation module 60 outputs a pulse signal when the dynamic voltage reaches the dynamic threshold.

[0025] In some embodiments of this application, the comparison module 50 is mainly used as a circuit unit for performing voltage signal comparison, and can be a commercially available conventional operational amplifier or voltage comparator integrated chip, such as the LM393. The pulse generation module 60 is a circuit unit for performing level transitions and generating a signal with a specified pulse width. The memristor element 30 and the first impedance element 40 are connected in series to form a first signal path; the first sensing element 10 and the second impedance element 20 are connected in series to form a second signal path. The first signal path and the second signal path are respectively used to connect to the first input terminal and the second input terminal of the comparison module 50, so that the comparison module 50 receives dynamic voltage and dynamic threshold respectively, and triggers the pulse generation module 60 based on the relationship between dynamic voltage and dynamic threshold, so that the pulse generation module 60 outputs the corresponding pulse signal. The memristor element 30 is preferably a memory resistor, which is the core device of the spiking neural network. It can efficiently encode the analog input signal into a pulse sequence. The memristor element 30 can accumulate input charge and discharge to output a pulse when the threshold is reached. The memristor element 30 can play the role of resistance change in the circuit, thereby simulating the variability of the membrane conductance of biological neurons.

[0026] As an optional implementation, the first impedance element 40 and the second impedance element 20 can be conventional fixed-value carbon film resistors or metal film resistors. As a further preferred embodiment, to ensure long-term reliability in automotive environments, the memristor element 30 is made of highly stable niobium oxide (NbO2), exhibiting excellent electrical consistency with batch-to-batch parameter variations of less than 5%, and requiring cycle durability greater than 10^8 cycles.

[0027] In actual operation, the memristor element 30 is used to receive a first-mode signal, such as an electrical signal converted from physical quantities like radar echo or range signal. Its internal conductance changes with the input continuous or discrete electrical signal, thereby forming a continuously accumulating or changing dynamic voltage at the common terminal with the first impedance element 40. Figure 2 The state diagram of the Vmem node and voltage changes shows that because the resistance of the memristor element 30 gradually decreases, the voltage drop across the first impedance element 40 gradually increases, causing the voltage at the Vmem node to gradually rise. Figure 2 It is understood that the dynamic voltage is input as an integral potential to the first input terminal of the comparison module 50. Simultaneously, the first sensing element 10 is configured to directly expose to or receive a second-mode signal from the external environment, such as a specific environmental physical quantity, and changes its resistivity in real time according to the strength of the second-mode signal. Through the voltage division effect with the second impedance element 20, the potential at its common terminal fluctuates in real time, thereby continuously outputting a dynamic threshold to the second input terminal of the comparison module 50.

[0028] Through the above structural configuration, this embodiment abandons the traditional scheme of setting the comparison reference with a fixed voltage divider resistor, and also abandons the original sliding rheostat to change the dynamic threshold of the input terminal of the comparison module 50. This makes the judgment reference (dynamic threshold) of the comparison module 50 no longer a constant, but directly controlled by the real-time signal strength of the second mode signal, realizing the electrical mapping of environmental information to the judgment reference.

[0029] In this embodiment, during operation, the first modal signal is continuously input to the memristor element 30, causing the dynamic voltage at the memristor element 30 to gradually accumulate and change. Simultaneously, based on changes in the external environment, the second modal signal gradually fluctuates, causing the dynamic threshold of the comparison module 50 to float in real time. The comparison module 50 continuously monitors the relationship between the dynamic voltage and the dynamic threshold. When the absolute amplitude of the dynamic voltage reaches and exceeds the real-time floating dynamic threshold, the output of the comparison module 50 undergoes a level flip, triggering the subsequent pulse generation module 60 to output a pulse signal with predetermined characteristics. This embodiment achieves the comparison and pulse triggering of two different modal signals at a single neuron node.

[0030] As a safe and equivalent alternative implementation, in addition to being configured as a photosensor, those skilled in the art can also use a thermistor or varistor to sense ambient temperature or pressure as a second mode signal. The first sensing element 10 here is mainly selected based on different environments and applications to determine the dynamic threshold. Furthermore, the active layer material of the memristor element 30 can also be tantalum oxide (TaOx) with higher thermal stability or hafnium oxide (HfOx) compatible with standard semiconductor processes.

[0031] like Figure 1 and Figure 2 As shown, the pulse generation module 60 further includes a timing element 70; the pulse generation module 60 has a discharge terminal and a trigger terminal, the trigger terminal is connected to the output terminal of the comparison module 50, and the discharge terminal is connected to the timing element 70.

[0032] In one embodiment of this example, the timing element 70 is coupled to the charging and discharging circuit of the pulse generation module 60. When the trigger terminal of the pulse generation module 60 receives the level flip signal from the comparison module 50, the pulse generation module 60 starts the pulse output state and simultaneously activates its internal or associated discharge terminal. The timing element 70 intervenes in the charge discharge or accumulation of the discharge terminal to adjust the pulse width of the pulse output by the pulse generation module 60.

[0033] In this embodiment, the pulse generation module 60 is defined as a component with specific terminals, and a timing element 70 is introduced, so that the pulse triggering action and the pulse duration are separately controlled at the electrical port. The pulse generation module 60 can be selected from corresponding chips, or it can be implemented by building a pulse triggering circuit. In order to consider the miniaturization of the circuit, this embodiment preferably uses a 555 timer.

[0034] like Figure 1 and Figure 2 As shown, the timing element 70 includes a second sensing element 71 and an energy storage element 72. The second sensing element 71 and the energy storage element 72 are connected in series, and the common terminal of the second sensing element 71 and the energy storage element 72 is connected to the discharge terminal of the pulse generation module 60. The second sensing element 71 is used to receive the second mode signal and change its own resistance according to the second mode signal to adjust the width of the pulse signal.

[0035] In one embodiment of this invention, the second sensing element 71 and the first sensing element 10 may be the same device, or simple parameter adjustments may be made according to the requirements of the external environment; the energy storage element 72 is mainly used to store charge and is connected to the discharge terminal of the pulse generation module 60 to maintain the pulse width of the pulse signal output by the pulse generation module 60.

[0036] In this embodiment, when in operation, the second sensing element 71 and the first sensing element 10 are placed in the same environmental field or receive the second mode signal from the same signal source. When the output level of the comparison module 50 flips, the pulse generation module 60 is triggered, and the energy storage element 72 charges and discharges through the second sensing element 71. Since the resistance of the second sensing element 71 changes in real time with the second mode signal, the time constant (RC constant) of the charging and discharging circuit is dynamically changing accordingly. The discharge terminal of the pulse generation module 60 determines the end time of the pulse period by monitoring the charging and discharging levels at both ends of the energy storage element 72.

[0037] Compared to the basic scheme, this embodiment enables the neuron to output not only monotonous level transitions, but also allows the width of its output pulse signal to adaptively shorten or lengthen in real time according to external environmental characteristics (second modal signal), increasing the dimension of information encoding. Simultaneously, the resistance of the first sensing element 10 also changes with the second modal signal, adjusting the dynamic threshold at the input of the comparison module 50. When the first modal signal (such as the dynamic voltage corresponding to radar distance) and the second modal signal (such as ambient light intensity) are input simultaneously, not only is the fusion of different modal signals achieved, but also coordinated correlation modulation is realized through the first sensing element 10 and the second sensing element 71. For example, in extremely dark conditions and at close range, the first sensing element 10 lowers the dynamic threshold, causing the pulse generation module 60 to trigger earlier, ensuring a low-delay response. Simultaneously, the second sensing element 71 adjusts the pulse width to change the frequency characteristics, forming a dual regulation of threshold and pulse width, simulating the stress response of a biological system under extreme danger—something that traditional linear superposition circuits cannot achieve.

[0038] In this embodiment, the first sensing element 10 controls the voltage threshold and the second sensing element 71 controls the capacitor discharge time. The two sense the second mode signal together at different circuit nodes, which effectively filters the electrical noise of a single node and ensures the stability and reliability of the output pulse signal.

[0039] In a specific example, the pulse width can be adjusted in the range of 1 to 100 milliseconds by using such components.

[0040] In a specific application scenario, this embodiment can be applied to a perception and collision avoidance system for advanced intelligent driving. For example, the first modal signal is selected as the distance signal returned by millimeter-wave radar; the closer the distance signal, the faster the dynamic voltage rises. The second modal signal is selected as the ambient light intensity; for example, the ambient light intensity will be dim at night or when entering a tunnel. When the vehicle is driving at night, the second modal signal strength is weaker. The first sensing element 10 adaptively lowers the trigger threshold, while the second sensing element 71 changes the pulse width, sending a warning pulse with a special bandwidth for nighttime driving to the backend. The system does not need to first call the camera's computing power to identify that it is nighttime; the neurons directly issue a high-priority emergency braking pulse based on the dim light and close distance.

[0041] Of course, in another embodiment of this invention, this solution can also be applied to other scenarios, such as temperature and pressure health monitoring in wearable devices. For example, the first modal signal is the blood pressure or pulse signal transmitted by the flexible sensor, and the second modal signal is the body surface temperature. When a person has a fever, it is often accompanied by changes in heart rate. The first sensing element 10 and the second sensing element 71 respectively select two thermistors to sense the body surface temperature. The first sensing element 10 changes the integration threshold of the pulse signal; the second sensing element 71 changes the output pulse width, so that a pulse warning sequence with high-temperature characteristics can be sent to the outside world without waking up the main control Bluetooth chip under low power consumption.

[0042] like Figure 2 As shown, the energy storage element 72 is an integrating capacitor; wherein, the capacitance value of the integrating capacitor is negatively correlated with the rate of change of the first modal signal into the dynamic voltage.

[0043] In one embodiment of this invention, a conventional capacitor is selected as the energy storage element 72, and the value of the integrating capacitor defines the time accumulation characteristics of the neuron for the input signal. In this embodiment, the capacitance value of the integrating capacitor is set to be negatively correlated with the rate of change of the first modal signal into the dynamic voltage. The faster the rate of change of the first modal signal, the smaller the capacitance value of the integrating capacitor is used, which reduces the integration time constant. This allows the neuron circuit to have higher sensitivity to rapidly changing signal components, making it suitable for processing visual features; conversely, a larger capacitance value is used to smooth high-frequency components, making it suitable for processing slowly changing information such as temperature and pressure.

[0044] like Figure 1 and Figure 2 As shown, both the first sensing element 10 and the second sensing element 71 include photoresistors.

[0045] In one embodiment of this invention, the first sensing element 10 and the second sensing element 71 are preferably both set as photoresistors. The photoresistors can utilize the photoelectric effect of semiconductor materials. In the absence of light or weak light environment, the photoresistors exhibit a high resistance state; under strong light irradiation, the carrier concentration increases, causing the resistance value to drop rapidly.

[0046] This embodiment preferably uses a photoresistor, which enables the neuron circuit to directly convert spatial light intensity into impedance parameters, achieving low latency of visual modal information, and is particularly suitable for the aforementioned advanced driving perception and collision avoidance system.

[0047] like Figure 1 and Figure 2 As shown, both the first sensing element 10 and the second sensing element 71 are configured such that their resistance values ​​are negatively correlated with the intensity of the second modal signal; the resistance value of the first sensing element 10 decreases as the second modal signal increases, thereby reducing the dynamic threshold; the resistance value of the second sensing element 71 decreases as the second modal signal increases, thereby reducing the width of the pulse signal.

[0048] In one embodiment of this example, the resistance change relationship between the first sensing element 10 and the second sensing element 71 is defined, indicating that in Figure 2 In the circuit, regardless of the physical form of the second mode signal, such as light intensity, pressure, humidity, temperature, or electromagnetic fields, the corresponding second sensing element 71 can be selected as a photoresistor, piezoresistive resistor, humidity-sensitive resistor, thermistor, or magnetoresistive resistor. It is only necessary to ensure that the resistance of the second sensing element 71 gradually decreases as the intensity of the second mode signal increases. Figure 2 It is known that when the resistance of the second sensing element 71 decreases, the dynamic threshold will gradually decrease in order to provide dynamic voltage response speed; the embodiments of this application can be applied to different fields and scenarios for measuring different physical quantities.

[0049] In this embodiment, the second sensing element 71 is a photoresistor. When the light intensity of the second mode signal increases, the resistance of the first sensing element 10 decreases, which changes the voltage divider node potential and causes the dynamic threshold of the second input terminal of the comparison module 50 to decrease. At this time, the weak radar distance signal of the first mode signal can easily reach the dynamic threshold. Simultaneously, the resistance of the second sensing element 71 decreases, which reduces the time constant of the charging and discharging circuit and shortens the width of the pulse signal and the subsequent recovery period.

[0050] The negative correlation configuration in this embodiment enables the neuron circuit to exhibit an extremely low trigger threshold and extremely fast recovery speed when multiple danger signals occur concurrently, resulting in a nonlinear surge in output frequency. Test results show that when the information of increasing light intensity and decreasing distance are synchronized in time, the firing frequency of the neuron can be increased several times, effectively breaking through the processing limit of conventional linear superposition.

[0051] like Figure 1 and Figure 2 As shown, the non-common terminal of the first impedance element 40 and the non-common terminal of the first sensing element 10 are both connected to the ground terminal, and the non-common terminal of the second impedance element 20 is connected to the power supply terminal; the inverting input terminal of the comparison module 50 serves as the first input terminal to monitor the dynamic voltage; the non-inverting input terminal of the comparison module 50 serves as the second input terminal to obtain the dynamic threshold.

[0052] In one embodiment of this invention, the second impedance element 20 and the first sensing element 10 form a pull-up voltage divider network through the aforementioned connection configuration between the ground and power supply terminals. The inverting input terminal of the comparator module 50 is connected to the voltage integration node of the memristor element 30, and the non-inverting input terminal is connected to the reference level node of the pull-up voltage divider network. This specific circuit structure ensures excellent common-mode rejection capability throughout the entire operating voltage range, such as from 3.3V to 5V, during the comparison process between the dynamic voltage and the dynamic threshold, significantly reducing the probability of false triggering caused by power supply ripple.

[0053] like Figure 2 As shown, this embodiment also includes a pull-up resistor R1; the output terminal of the comparison module 50 is connected to the power supply terminal via the pull-up resistor R1, and the output terminal of the comparison module 50 is the low-level trigger node of the pulse generation module 60.

[0054] In one embodiment of this invention, the comparator module 50 employs an open-collector structure, meaning it relies on an external power supply and pull-up resistor R1 to output a valid high-level signal. When the dynamic voltage at the inverting input terminal does not reach the dynamic threshold at the non-inverting input terminal, the open-collector is cut off, while the pull-up resistor R1 provides a stable high level, keeping the trigger terminal of the pulse generation module 60 continuously at a high level. Figure 2 The TRI pin of the pulse generator module 60 is pulled high; once the dynamic voltage exceeds the dynamic threshold, the output of the comparator module 50 is pulled low. Figure 2 The TRI pin of the pulse generator module 60 is pulled low. This embodiment ensures that the signal edges transmitted to the trigger node of the pulse generator module 60 are steep, eliminating pulse signal trigger jitter caused by floating.

[0055] This application also discloses a spiking neural network, which includes a multimodal signal input interface and a neuron array. The multimodal signal input interface is configured to receive a first modal signal and a second modal signal in parallel. The neuron array includes multiple multimodal artificial neuron circuits as described above. The multiple multimodal artificial neuron circuits receive and fuse the first modal signal and the second modal signal through the multimodal signal input interface.

[0056] In one embodiment of this invention, the underlying sensor interface acquires different modal signals from the external environment in parallel through a multimodal signal input interface. After receiving the different modal signals, an array composed of multiple artificial neuron circuits performs integration, threshold determination, and bandwidth modulation internally, outputting an encoded spatial pulse sequence. Based on this, an FPGA decision layer can be configured to perform pulse frequency analysis and intelligent decision-making; for example, a multi-level decision-making strategy based on a state machine can be adopted: the first level checks the safety status and triggers an emergency response when a specific channel frequency exceeds the limit, such as triggering an emergency response when the frequency exceeds 50Hz; the second level analyzes environmental characteristics and calculates the channel frequency correlation coefficient; the third level generates the final control strategy and automatically performs decision-making activities to ensure real-time response.

[0057] The following is an explanation and illustration of the specific waveforms: like Figure 3 The diagram shows the basic electrical performance of the memristor 30, with curves illustrating its hysteresis loop characteristics. In the arrow 1 stage, the voltage is low and the current is minimal, indicating that the memristor 30 exhibits a high-resistivity state. In the arrow 2 stage, the voltage of the memristor 30 reaches a threshold value of approximately 1.0V, resulting in a significant jump in current and an increase in conductance. Arrow 3 illustrates the state maintenance or recovery of the memristor 30 after power is turned off. Figure 3 The displayed voltages, such as 2.5V and 2.7V, all follow this trend.

[0058] Figure 4 The graph shows the current repetition of the memristor element 30 under multiple sets of identical voltages. In 100 consecutive tests, the IV curves highly overlapped with almost no divergence. Therefore, Figure 4 This indicates that existing technologies suffer from poor device robustness due to process fluctuations. This application solves the problem of poor robustness of traditional circuits by achieving excellent electrical consistency with parameter differences of less than 5% within batches.

[0059] Figure 5 and Figure 6 The plot shows the current of memristor 30 versus pulse width / amplitude. When a continuous square wave pulse is applied, the current of memristor 30 does not reach its maximum instantaneously, but rather gradually increases with the increase of the pulse number. The larger the pulse width (W) or the higher the amplitude (V), the greater the current ramp-up slope. Figure 5 and Figure 6 It was confirmed that the memristor 30 has the function of leakage integration, which enables the first mode signal to be continuously input into the memristor 30, so that the dynamic voltage (i.e. node Vmem) at the memristor 30 terminal gradually accumulates and changes.

[0060] Figure 7 The waveform timing diagram of each node of the LIF neuron is shown. It has three input terminals, namely Vinput, Vmemristor, and Vcompare. Vinput refers to the left input terminal of the memristor element 30. Vmemristor refers to the first input terminal of the comparator module 50, which is the common terminal of the memristor element 30 and the first impedance element 40, and its output dynamic voltage.

[0061] Vcompare refers to the output terminal of the comparator module 50, that is, the node voltage at the TRI pin where the comparator module 50 is connected to the pulse generator module 60.

[0062] Figure 7 The entire integration and triggering process is demonstrated. The discrete input of the Vinput signal causes the node Vmem of the Vmemristor to rise in a step-like manner, achieving integration accumulation. When the node Vmem rises to a certain baseline, Vcompare is instantly pulled down from a high level of 3.3V to a low level of 0V, completing the pulse triggering action. This indicates that when the absolute amplitude of the dynamic voltage reaches and exceeds the real-time floating dynamic threshold, the output of the comparison module 50 undergoes a level flip, triggering the subsequent pulse generation module 60.

[0063] Figure 8 The graph shows the width of the output pulse signal as a function of resistance. The resistance values ​​in the graph represent the voltage waveform of the pulse signal over time when the second sensing element 71 selects different resistance values ​​according to the second mode signal. Specifically: When the resistance of the second sensing element 71 decreases from large to small, such as Figure 8 As shown in the figures of 300k, 100K, 47K and 20k, the pulse width of the circuit output becomes significantly narrower, indicating that the pulse signal emission frequency increases significantly. This corresponds to the fact that the resistance of the second sensing element 71 mentioned in this application decreases as the second mode signal is enhanced, so as to reduce the width of the pulse signal. This embodiment shows that the circuit can map the nonlinear characteristics of the external environment (such as the resistance change caused by the change in light intensity) into the pulse width characteristics of the pulse.

[0064] In the above embodiments, this embodiment also connects a third sensing element (such as...) in series between the memristor element 30 and the input terminal of the first mode signal. Figure 2The third sensing element LDR3 is used to receive the second mode signal (such as ambient light intensity) and dynamically adjust its own resistance value according to the intensity of the second mode signal. The third sensing element LDR3 and the memristor element are connected in series to form a voltage divider network to regulate the actual driving voltage applied across the memristor element 30, thereby dynamically changing the rate of change of the conductance of the memristor element 30.

[0065] In intelligent driving scenarios, when the ambient light intensity is low (such as at night or inside a tunnel), the third sensing element LDR3 exhibits a high-impedance state. At this time, even if there is a weak radar noise signal at the input, due to the high impedance voltage division effect of the third sensing element, the voltage applied to the memristor element 30 is extremely small, and the memristor element 30 exhibits a weak response or no response state, forming an adaptive noise threshold that effectively blocks the integration of invalid signals and greatly reduces the static power consumption of the memristor element 30.

[0066] Conversely, when the ambient light intensity is high, the resistance of the third sensing element drops rapidly, and the memristor element 30 receives an effective driving voltage, resulting in a significant response. The power consumption adaptive adjustment method in this embodiment makes the energy efficiency of the neural circuit significantly better than that of traditional continuous integration neural circuits in complex environments.

[0067] This application achieves multidimensional nonlinear modulation of neuronal dynamics through the coordinated operation of the first sensing element 10, the second sensing element 71, and the third sensing element LDR3. When the ambient light intensity suddenly increases and a dangerous distance is input, the resistance of the third sensing element LDR3 decreases, leading to an increase in the input current, which causes a sharp increase in the dynamic voltage (membrane potential) integration rate of the memristor element 30. Simultaneously, the resistance of the first sensing element 10 decreases, causing the dynamic threshold of the comparison module 50 to be actively lowered. In addition, the resistance of the second sensing element 71 decreases, shortening the discharge cycle of the pulse generation module. This embodiment combines multiple couplings of accelerated integration, lowered threshold, and shortened discharge cycle, enabling the neuronal circuit of this embodiment to output pulse sequences at an exponentially increasing firing frequency when responding to high-priority multimodal danger signals. This produces a stress-induced hypersensitive response that cannot be achieved by traditional linear superposition circuits, simulating the hardware fusion of biological vision and spatial joint perception.

[0068] Specifically, such as Figure 9 As shown: Figure 9 The diagram illustrates the current-time (It) response curves of the memristor element 30 under different equivalent driving voltages, which are controlled by the voltage divider effect of the third sensing element LDR3. Specifically, when a first-mode signal of fixed strength is applied to the input terminal, the third sensing element LDR3 dynamically changes its resistance value according to the strength of an external second-mode signal (such as ambient light), which can be categorized into the following states: 1. Low Light Suppression State: When the ambient light intensity is low, the third sensing element LDR3 exhibits a high impedance state, significantly reducing the input voltage load. Figure 9 As shown in the 5.6V curve, this curve represents the actual voltage applied across the memristor element 30. The growth of the conductive filaments inside the memristor element 30 is suppressed, and the current baseline is at... At the following extremely low levels, the current value rises slowly as the circuit integrates over time. In this state, the circuit effectively filters background noise while also reducing the device's static power consumption.

[0069] 2. Strong photosensitization state: When the intensity of external ambient light increases, the resistance of the third sensing element LDR3 drops rapidly. For example... Figure 9 As shown in the 6.4V and 7V curves, this curve represents the actual driving voltage obtained across the memristor element 30. Compared to the 5.6V curve, the 6.4V and 7V curves show a significant increase in current, indicating accelerated carrier migration and the formation of conductive filaments. The current response exhibits a slow upward trend, and with a significant increase in the integration rate, the maximum current can reach approximately [value missing]. A and A.

[0070] The above test data objectively demonstrates that the introduction of the third sensing element LDR3 enables this neuron circuit to break free from the limitation of constant integration rate. The circuit can actively switch between sleep and sensitization states according to environmental characteristics, thereby achieving the best balance between low power consumption and high sensitivity multimodal fusion. Of course, the external ambient light intensity in this embodiment can also be replaced with various different environmental parameters, such as temperature and pressure.

[0071] This application also discloses an electronic device, which includes the aforementioned spiking neural network.

[0072] In one embodiment of this invention, the electronic device can be used in a variety of different scenarios, such as electronic devices in multiple fields such as advanced intelligent driving, medical devices, visual recognition, radar, and ultrasound. It is only necessary to adjust the first mode signal and the second mode signal, and replace the corresponding second sensing element 71.

[0073] Working principle: When the electronic device is powered on, the front-end sensors capture first-mode signals and second-mode signals in parallel, such as radar distance and visual light intensity. The first-mode signal flows through an integration network composed of a first impedance element 40 and a memristor element 30, and the dynamic conductivity characteristics of the memristor element 30 are used to convert the level into a gradually accumulating dynamic voltage. At this time, the second-mode signal is synchronously introduced into a voltage divider network composed of a first sensing element 10 and a second impedance element 20, so that the dynamic threshold of the comparator becomes a reference voltage that fluctuates in real time with the ambient light intensity. When the dynamic voltage approaches and exceeds the dynamic threshold, the output of the comparator module 50 is pulled low instantaneously, triggering the timer in the pulse generation module 60 to flip and output a pulse level. During the pulse maintenance phase, the second sensing element 71 adjusts the resistance value of the discharge circuit in real time according to the second-mode signal, accelerating or delaying the discharge process of the energy storage element 72. In the charging and discharging circuit, the adaptive decrease in the resistance value of the second sensing element 71 and the shortening of the recovery period of the pulse generation module 60 are coupled and linked, ultimately accurately mapping the nonlinear characteristics of the external environment into a high-frequency or low-frequency discrete spatial pulse sequence.

[0074] The overall solution provided in this application does not rely on complex digital-to-analog conversion peripheral chips. It achieves the fusion of multi-modal signals solely through basic analog links, possessing extremely high robustness and low power consumption advantages, making it particularly suitable for the requirements of edge intelligence and high-reliability automotive scenarios.

[0075] The above description is merely a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural transformations made based on the inventive concept of this application and the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included within the patent protection scope of this application.

Claims

1. A multi-modal artificial neuron circuit, characterized by, include: Comparison module; A pulse generation module is connected to the output terminal of the comparison module; A memristor and a first impedance element are connected in series, and their common terminal is connected to the first input terminal of the comparison module. The memristor is used to receive a first mode signal and output a dynamic voltage to the first input terminal. A first sensing element and a second impedance element are connected in series, and their common terminal is connected to the second input terminal of the comparison module. The first sensing element is used to receive the second modal signal and output a dynamic threshold to the second input terminal. The pulse generation module outputs a pulse signal when the dynamic voltage reaches the dynamic threshold. The pulse generation module also includes a timing element; The pulse generation module has a discharge terminal and a trigger terminal. The trigger terminal is connected to the output terminal of the comparison module, and the discharge terminal is connected to the timing element. The timing element includes a second sensing element and an energy storage element, the second sensing element and the energy storage element are connected in series, and the common terminal of the second sensing element and the energy storage element is connected to the discharge terminal of the pulse generation module; The second sensing element is used to receive the second modal signal and change its own resistance according to the second modal signal to adjust the width of the pulse signal.

2. The multi-modal artificial neuron circuit of claim 1, wherein, The energy storage element is an integrating capacitor; The capacitance value of the integrating capacitor is configured to be negatively correlated with the rate of change of the first modal signal into the dynamic voltage.

3. The multi-modal artificial neuron circuit of claim 1 or 2, wherein, Both the first sensing element and the second sensing element include a photoresistor.

4. The multi-modal artificial neuron circuit of claim 1 or 2, wherein, Both the first sensing element and the second sensing element are configured such that their resistance values ​​are negatively correlated with the strength of the second mode signal; The resistance of the first sensing element decreases as the second modal signal increases, thereby reducing the dynamic threshold. The resistance of the second sensing element decreases as the second modal signal increases, thereby reducing the width of the pulse signal.

5. The multi-modal artificial neuron circuit of claim 1, wherein, Both the non-common terminal of the first impedance element and the non-common terminal of the first sensing element are connected to the ground terminal, and the non-common terminal of the second impedance element is connected to the power supply terminal. The inverting input terminal of the comparison module serves as the first input terminal to monitor the dynamic voltage; The in-phase input of the comparison module serves as the second input to obtain the dynamic threshold.

6. The multimodal artificial neuron circuit as described in claim 1, characterized in that, It also includes pull-up resistors; The output terminal of the comparison module is connected to the power supply terminal via the pull-up resistor, and the output terminal of the comparison module is the low-level trigger node of the pulse generation module.

7. The multimodal artificial neuron circuit as described in claim 1, characterized in that, The active layer material of the memristor element is any one of niobium oxide, tantalum oxide, or hafnium oxide.

8. The multimodal artificial neuron circuit as described in claim 1, characterized in that, The comparison module includes an operational amplifier or a voltage comparator integrated chip; The pulse generation module includes a 555 timer.

9. A spiking neural network, characterized in that, include: A multi-mode signal input interface is configured to receive the first mode signal and the second mode signal in parallel. A neuron array comprising a plurality of multimodal artificial neuron circuits as described in any one of claims 1 to 8, wherein the plurality of multimodal artificial neuron circuits receive and fuse the first modal signal and the second modal signal through the multimodal signal input interface.

10. An electronic device, characterized in that, Including the spiking neural network as described in claim 9.

Images