Image fusion method based on multiband response artificial visual pulse neuron array

Abstract

The invention discloses an image fusion method based on a multiband response artificial visual pulse neuron array, and the method comprises the steps: inputting a plurality of groups of optical images of different bands into the artificial visual pulse neuron array, carrying out the image preprocessing, converting the brightness values of different pixels into pulse signals, and carrying out the image fusion of the pulse signals; correspondingly obtaining a plurality of groups of sequence pulse signals; wherein the artificial visual spiking neuron array comprises a plurality of artificial visual spiking neurons; the artificial visual spiking neuron comprises a photoelectric sensor and a threshold switch memristor; inputting the plurality of groups of sequence pulse signals into an artificial visual pulse neural network, and performing fusion processing by combining response features of different wavebands to obtain a fused pulse image; and decoding the fused pulse image into a pixel image to obtain a final fused image. According to the method, multiband image fusion is realized by using the artificial visual pulse neural network, and the quality and efficiency of image fusion are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image fusion, and in particular relates to an image fusion method based on a multi-band response artificial visual pulse neuron array. Background Art

[0002] Image fusion is commonly used to combine image information from different sources or different bands to obtain an image that contains more information. In the fields of remote sensing imagery and medical imaging, high-quality fusion of multi-source image data is of great significance.

[0003] Traditional image fusion methods, such as wavelet transform and principal component analysis, suffer from low image resolution and poor fusion performance in some application scenarios. With the rapid development of artificial intelligence (AI), spiking neural networks (SNNs), a computational model that mimics biological neurons, have demonstrated high efficiency and accuracy in processing complex signals and image data. However, existing SNNs are primarily used for single-band image processing, and no technology has yet been developed for multi-band image fusion. Summary of the Invention

[0004] To address the above-mentioned problems in the prior art, the present invention provides an image fusion method based on a multi-band response artificial visual pulse neuron array. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0005] In a first aspect, the present invention proposes an image fusion method based on a multi-band response artificial visual pulse neuron array, comprising:

[0006] Several groups of optical images of different wavelengths are input into an artificial visual spiking neuron array for image preprocessing to convert the brightness values ​​of different pixels into pulse signals, thereby obtaining several groups of corresponding sequence pulse signals. The artificial visual spiking neuron array includes multiple artificial visual spiking neurons, each of which includes a photosensor and a threshold switching memristor. The photosensor is used to sense optical signals of multiple wavelengths and convert the optical signals into current signals. The memristor is used to encode the current signals into pulse signals.

[0007] Input several groups of sequential pulse signals into the artificial visual pulse neural network, combine the response characteristics of different bands to perform fusion processing, and obtain the fused pulse image;

[0008] The fused pulse image is decoded into a pixel image to obtain the final fused image.

[0009] In a second aspect, the present invention proposes an image fusion device based on a multi-band response artificial visual pulse neuron array, for implementing the method proposed in the first aspect of the present invention, the device comprising:

[0010] An image preprocessing module, configured with an artificial visual pulse neuron array, is used to perform image preprocessing on a plurality of optical images of different wavelengths, thereby converting the brightness values ​​of different pixels into pulse signals, thereby obtaining a plurality of corresponding sequence pulse signals. The artificial visual pulse neuron array includes a plurality of artificial visual pulse neurons, each of which includes a photoelectric sensor and a threshold switching memristor. The photoelectric sensor is used to sense optical signals of multiple wavelengths and convert the optical signals into current signals. The memristor is used to encode the current signals into pulse signals.

[0011] The image fusion module is equipped with an artificial visual pulse neural network, which is used to fuse several groups of sequential pulse signals by combining the response characteristics of different bands to obtain a fused pulse image;

[0012] The image decoding module is used to decode the fused pulse image into a pixel image to obtain the final fused image.

[0013] Beneficial effects of the present invention:

[0014] 1. The image fusion method based on the multi-band response artificial visual pulse neuron array proposed in the present invention pre-designs an artificial visual pulse neuron including a photoelectric sensor and a threshold switch memristor, wherein the photoelectric sensor is used to sense light signals of multiple bands and convert the light signals into current signals; the memristor is used to encode the current signal into a pulse signal; then the artificial visual pulse neuron array is used to pre-process several groups of optical images of different bands to convert the brightness values ​​of different pixels into pulse signals, corresponding to several groups of serial pulse signals; then the artificial visual pulse neural network is used to fuse the several groups of serial pulse signals to obtain a fused pulse image; finally, the fused pulse image is decoded into a pixel image, thereby realizing image fusion. This method realizes image fusion using a pulse neural network. Through the temporal learning and response characteristics of the pulse neural network, it can retain more image details, reduce information loss, and improve the quality of image fusion. The generated image has high resolution and high contrast, and is suitable for the fusion of multi-source image data, especially in the fields of remote sensing images and medical imaging. It has important application value.

[0015] 2. The artificial visual spiking neuron in the present invention consists of only a photoelectric sensor and a threshold switch memristor integrated unit. Using only these two devices, it can successfully simulate the function of biological visual spiking neurons. It has a simple structure and stable performance, greatly reducing the large footprint and high computational cost of traditional artificial visual spiking neuron circuits, and laying a solid foundation for the construction of integrated and low-power artificial vision systems.

[0016] 3. The artificial visual pulse neurons designed in this invention can not only perceive optical signals of different wavelengths in real time, but also convert them into electrical signals. They can efficiently handle a large number of complex visual perception tasks, and their performance is superior to that of traditional dynamic vision sensors.

[0017] 4. The threshold switching memristor in the artificial visual pulse neuron of the present invention has excellent stability and can maintain consistent performance during long-term use, ensuring reliable conductance control and efficient computing power.

[0018] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of an image fusion method based on multi-band response artificial visual pulse neurons provided by an embodiment of the present invention;

[0020] Figure 2 This is another flow chart of an image fusion method based on multi-band response artificial visual pulse neurons provided by an embodiment of the present invention;

[0021] Figure 3 1 is a schematic diagram of the structure of an artificial visual pulse neuron provided by an embodiment of the present invention;

[0022] Figure 4 The UV-visible-infrared absorption spectra of the indium oxide film and the PY-IT film provided in the embodiments of the present invention;

[0023] Figure 5 This is a graph of current pulse output of an artificial visual pulse neuron provided by an embodiment of the present invention under ultraviolet 365nm;

[0024] Figure 6 This is a graph of the current pulse output of the artificial visual pulse neuron provided by an embodiment of the present invention under near-infrared light of 800nm;

[0025] Figure 7 is another structural schematic diagram of an artificial visual pulse neuron provided by an embodiment of the present invention;

[0026] Figure 8 This is a near-infrared and ultraviolet image fusion effect diagram provided by an embodiment of the present invention;

[0027] Figure 9 This is a structural block diagram of an image fusion device based on a multi-band response artificial visual pulse neuron array provided by an embodiment of the present invention;

[0028] Description of reference numerals:

[0029] 1-Silicon substrate; 2-Silicon dioxide dielectric layer; 3-Photosensitive layer; 31-Semiconductor layer; 32-Organic layer; 4-Source and drain electrodes; 5-Bottom electrode layer; 6-Middle layer; 7-Top electrode layer. DETAILED DESCRIPTION

[0030] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0031] Mott memristors exhibit threshold switching, meaning they exhibit a sudden change in conductance at a specific voltage. This property is well-suited for simulating the "on-off" behavior of neurons, where neurons trigger a "pulse" or "action potential" only when the input signal reaches a certain intensity. This property makes Mott memristors more efficient for use in neural networks. Combined with their low power consumption, high efficiency, and high integration density, they hold enormous potential for application in neuromorphic network computing and artificial intelligence hardware. Therefore, further development of artificial visual spiking neurons based on Mott memristors is of great research significance.

[0032] Based on this, the first aspect of the present invention designs an artificial visual pulse neuron with multi-band response function based on the Mott memristor, and accordingly proposes an image fusion method based on a multi-band response artificial visual pulse neuron array.

[0033] Please see the joint Figure 1 and Figure 2 , Figure 1 This is a flow chart of an image fusion method based on multi-band response artificial visual pulse neurons provided by an embodiment of the present invention. Figure 2 This is another flow chart of an image fusion method based on multi-band response artificial visual pulse neurons provided by an embodiment of the present invention; the method mainly includes the following steps:

[0034] Step 1: Input several groups of optical images of different wavelengths into the artificial visual pulse neuron array and perform image preprocessing to convert the brightness values ​​of different pixels into pulse signals, thereby obtaining several groups of corresponding sequence pulse signals.

[0035] Among them, the artificial visual pulse neuron array includes multiple artificial visual pulse neurons; the artificial visual pulse neuron includes a photoelectric sensor and a threshold switching memristor; the photoelectric sensor is used to sense light signals in multiple bands and convert the light signals into current signals; the memristor is used to encode the current signal into a pulse signal.

[0036] Step 2: Input several groups of sequential pulse signals into the artificial visual pulse neural network, combine the response characteristics of different bands to perform fusion processing, and obtain the fused pulse image.

[0037] Step 3: Decode the fused pulse image into a pixel image to obtain the final fused image.

[0038] It is understandable that, in this embodiment, it is first necessary to pre-design the artificial visual pulse neuron structure according to actual needs.

[0039] For details, see Figure 3 , Figure 3 : is a structural diagram of an artificial visual pulse neuron provided by an embodiment of the present invention, wherein the artificial visual pulse neuron includes a silicon substrate 1, a silicon dioxide dielectric layer 2, a photosensitive layer 3, a source and drain electrode 4, a bottom electrode layer 5, an intermediate layer 6, and a top electrode layer 7; wherein,

[0040] A silicon dioxide dielectric layer 2 is located on a silicon substrate 1;

[0041] The photosensitive layer 3 is located on the upper left surface of the silicon dioxide dielectric layer 2;

[0042] Source and drain electrodes 4 are located on either side of the photosensitive layer 3 and extend to the lower left or lower right onto the silicon dioxide dielectric layer 2, forming a photosensor on the left side of the entire neuron structure. Silicon substrate 1 serves as both the supporting substrate and gate of the photosensor. The photosensitive layer 3 has different absorption capacities for light of different wavelengths, enabling the photosensor to have multi-band sensing capabilities.

[0043] The bottom electrode layer 5 is located on the upper right surface of the silicon dioxide dielectric layer 2; the middle layer 6 completely covers the bottom electrode layer 5 and extends to both sides of the silicon dioxide dielectric layer 2 in a step-like manner; the top electrode layer 7 completely covers the middle layer 6 and extends to both sides of the silicon dioxide dielectric layer 2 in a step-like manner to form a threshold switching memristor on the right side of the entire neuron structure.

[0044] It is understandable that for artificial visual spiking neurons to encode and output pulse signals, the channel resistance of the photosensor must be between the high-resistance and low-resistance states of the threshold-switching memristor. In other words, when the photosensor device is in the non-conducting state, the threshold-switching memristor is in the high-resistance state; as the gate voltage gradually increases, the channel resistance of the photosensor device gradually decreases until the source current reaches a critical threshold, at which point the threshold-switching memristor is in the low-resistance state.

[0045] It should be noted that the photosensitive layer 3 of the photoelectric sensor with multi-band sensing function can be realized by a combination of several materials that can sense different bands, or by a single material that can sense different bands. Both methods can enable the artificial visual pulse neurons to have multi-band sensing function.

[0046] Optionally, as an implementation, such as Figure 3 In the artificial visual pulse neuron structure shown, the photosensitive layer 3 includes a semiconductor layer 31 and an organic layer 32; the semiconductor layer 31 is located on the upper left surface of the silicon dioxide dielectric layer 2; and the organic layer 32 is located in the middle of the upper surface of the semiconductor layer 31.

[0047] The present invention adopts Figure 3 The structure shown can utilize a combination of two materials that can sense different wavelengths to realize artificial visual pulse neurons that can sense two different wavelengths.

[0048] Optional, as Figure 3 In one implementation of the structure shown, the material of the semiconductor layer 31 is an indium oxide film with a thickness of 7-10 nm, preferably 8 nm; the material of the organic layer is a PY-IT film with a thickness of 80-120 nm, preferably 100 nm.

[0049] For details, see Figure 4 , Figure 4 The UV-visible-infrared absorption spectra of the indium oxide film and the PY-IT film provided by the embodiments of the present invention are shown. The photosensor exhibits significant absorption peaks in the near-infrared region around 800 nm and below 360 nm, indicating that the device can absorb light in both wavelengths.

[0050] When the artificial visual spiking neurons are exposed to near-infrared light, photogenerated electrons in the photosensor are injected from the lowest unoccupied molecular orbital into the conduction band of indium oxide, resulting in an increase in photocurrent, while photogenerated holes are blocked at the interface between indium oxide and the organic polymer layer. When exposed to ultraviolet light, neutral oxygen vacancy defects in the indium oxide film are converted into ionized oxygen vacancies, releasing free electrons, thereby increasing conductivity and leading to an increase in photocurrent.

[0051] Due to the threshold switching characteristics of the Mott memristor, the artificial visual spiking neuron i <R ch <R m Here, R ch represents the channel resistance of the photosensor, and R i and R m Represent the insulation resistance and metal resistance of the Mott memristor respectively. When the gate voltage and bias are fixed, R ch Determined by light intensity.

[0052] In this invention, pulse frequency refers to the number of pulses occurring per unit time and is usually used to describe the rate at which neurons fire. The first pulse time refers to the moment when the first pulse occurs, i.e., the arrival time of the first pulse in the signal.

[0053] Specifically, when light illuminates the artificial visual pulse neuron, the gate bias voltage (V GS =5V), a single electric pulse (V DS =5V, width of 20μs) to measure the encoded current pulse.

[0054] Figure 5 This is a graph of the current pulse output of the artificial visual pulse neuron under ultraviolet 365nm provided by an embodiment of the present invention; this graph shows the pulse behavior of the artificial visual pulse neuron under different ultraviolet light intensities. 2 Increased to 2.06mW / cm 2 When the pulse frequency increases monotonically from 1.05MHz to 1.75MHz, the first pulse time decreases from 2.51μs to 0.54μs.

[0055] Figure 6 This is a graph of the current pulse output of the artificial visual pulse neuron provided by the embodiment of the present invention under near infrared 800nm; this graph shows the pulse behavior of the artificial visual pulse neuron under different near infrared light intensities. 2 Increased to 185mW / cm 2 When the spike frequency increases monotonically from 1MHz to 1.75MHz, the first pulse time decreases from 4.92μs to 2.08μs.

[0056] It can be found that when the artificial visual spiking neurons are illuminated with light of different wavelengths, the artificial visual spiking neurons of the present invention can quickly and effectively represent rich pulse information under illumination of different optical power densities. These pulses can be measured at the bottom electrode contacts of the Mott memristor.

[0057] The present invention realizes an artificial visual pulse neuron that can simultaneously perceive the ultraviolet and near-infrared bands through the combination of indium oxide film and PY-IT film. Accordingly, the image fusion method based on the artificial visual pulse neuron can realize the fusion of near-infrared images and ultraviolet images.

[0058] Optionally, in another embodiment of the present invention, another artificial visual pulse neuron structure is provided, such as Figure 7 As shown, the photosensitive layer 3 only includes a semiconductor layer 31. The material of the semiconductor layer 31 is an indium selenide film with a thickness of 10-50nm, preferably 30nm. It can simultaneously perceive red, green and blue bands. Accordingly, the image fusion method based on the artificial visual pulse neuron can realize image fusion of red, green and blue bands.

[0059] The present invention adopts Figure 7 The structure shown can realize artificial visual pulse neurons with multi-band perception capabilities by using a material that can perceive different bands.

[0060] It is understandable that, in practical applications, other multi-band sensing functions can be achieved by replacing the materials of the semiconductor layer 31 and the organic layer 32 .

[0061] Regarding other components of the artificial visual pulse neuron structure designed in the present invention, the thickness of the silicon dioxide dielectric layer 2 is optionally 80-150 nm, preferably 100 nm; the thickness of the source and drain electrodes 4 is 40-100 nm, preferably 55 nm; the bottom electrode layer 5 and the top electrode layer 7 are both metal electrodes composed of titanium / platinum; and the intermediate layer 6 can be made of niobium oxide and have a thickness of 30-45 nm, preferably 40 nm. The specific thickness can be selected based on actual conditions.

[0062] The artificial visual spiking neuron in this invention consists of only a photoelectric sensor and a threshold-switching memristor integrated unit. Using only these two devices, it successfully simulates the function of biological visual spiking neurons. This simple structure and stable performance significantly reduce the large footprint and high computational costs of traditional artificial visual spiking neuron circuits, laying a solid foundation for building integrated and low-power artificial vision systems. The threshold-switching memristor exhibits excellent stability, maintaining consistent performance over extended periods of use, ensuring reliable conductance control and efficient computational power.

[0063] In addition, the artificial visual pulse neurons designed in the present invention can not only perceive optical signals of different bands in real time, but also convert them into electrical signals. They can efficiently process a large number of complex visual perception tasks, and their performance is better than that of traditional dynamic vision sensors.

[0064] After completing the structural design of the first artificial visual pulse neuron, it is arranged into an array, such as a 5*5 or 8*8 array, to pre-process the optical image.

[0065] Specifically, Figure 3 Taking the artificial visual pulse neuron structure shown in the figure as an example, it can process ultraviolet and near-infrared images. The pulse current signals of two different bands, ultraviolet and near-infrared, are input into the artificial visual pulse neural network to complete the image fusion task, specifically including:

[0066] Based on the pixel image, the artificial visual pulse neuron array is illuminated with light of different wavelengths under the same scene;

[0067] After sensing light from two wavelengths, the photoelectric sensor in the artificial visual spiking neuron converts the optical signal into an electrical signal. The threshold switching memristor in the artificial visual spiking neuron encodes the continuous electrical signal into two sets of sequential pulse signals.

[0068] The two sets of generated sequential pulse signals are input into the artificial visual pulse neural network, and intelligent fusion is performed based on the response characteristics of different bands to output the fused pulse image information;

[0069] Finally, the output pulse image information is decoded into a pixel image to form the final high-quality fused image, completing the image fusion process.

[0070] Furthermore, in this embodiment, the artificial visual pulse neural network adopts a conventional multi-layer pulse neuron structure, including an input layer, a hidden layer and an output layer; the neurons in each layer transmit information through pulses, and the connections between layers need to complete information transmission through adjustable weights.

[0071] In addition, before using the artificial visual pulse neural network to preprocess a number of groups of sequential pulse signals, the artificial visual pulse neural network needs to be trained.

[0072] Optionally, this embodiment can use supervised learning or unsupervised learning methods to train the artificial visual spiking neural network. The trained spiking neural network can synthesize multiple sequence pulse signals to generate a fused image, and obtain the pulse signal of the image in the output layer of the spiking neural network.

[0073] It is understandable that after obtaining the fused pulse image, the following steps are also included:

[0074] The fused pulse image is subjected to denoising and contrast enhancement operations to further improve the quality of the fused image.

[0075] Regarding the specific operation methods of image denoising and contrast enhancement, reference may be made to existing related technologies, which will not be described in detail in this embodiment.

[0076] Finally, the pulse signal output by the spiking neural network is decoded into image pixels. Common decoding methods include time decoding or frequency decoding. The detailed process can be referenced in existing related technologies.

[0077] The present invention proposes an image fusion method based on a multi-band responsive artificial visual spiking neuron array. The method pre-designs an artificial visual spiking neuron array comprising a photoelectric sensor and a threshold-switching memristor. The photoelectric sensor senses light signals in multiple bands and converts them into current signals; the memristor encodes the current signals into pulse signals. The artificial visual spiking neuron array then pre-processes several groups of optical images in different bands to convert the brightness values ​​of different pixels into pulse signals, resulting in several groups of sequential pulse signals. The artificial visual spiking neural network then fuses these groups of sequential pulse signals to obtain a fused pulse image. Finally, the fused pulse image is decoded into a pixel image, achieving image fusion. This method utilizes a spiking neural network to achieve image fusion. Through the temporal learning and response characteristics of the spiking neural network, it can retain more image details, reduce information loss, and improve the quality of image fusion. The resulting image has high resolution and high contrast, making it suitable for the fusion of multi-source image data and particularly valuable in remote sensing imagery and medical imaging.

[0078] To verify the effectiveness of the present invention, this example also uses a near-infrared image in which only the school logo can be seen, and an ultraviolet image in which only the school name can be seen, under the same scene. The method proposed in the present invention is used to perform an image fusion operation on the two images. The steps are as follows:

[0079] A near-infrared image and an ultraviolet image are input into an artificial visual pulse neuron array that responds to multiple wavelengths, generating two sets of pulse signals in different wavelengths.

[0080] Then the two sets of pulse signals are input into the artificial visual pulse neural network for image fusion, and the fused pulse image information is output;

[0081] Finally, the pulse image information is decoded into a pixel image to complete the image fusion process.

[0082] See Figure 8 , Figure 8This is a diagram showing the effect of near-infrared and ultraviolet image fusion provided by an embodiment of the present invention. As can be seen, the first near-infrared image shows the portrait, but the details of the house are unclear. The second ultraviolet image does not show the portrait, but the details of the house are clear. The fused image integrates information from the near-infrared and ultraviolet images, fully displaying the portrait and house details in both images, achieving image fusion with lower power consumption and lower efficiency.

[0083] The method proposed in the present invention can not only enhance the image fusion effect by improving the artificial visual pulse neural network, but also adjust the characteristics of the photoelectric sensor and the threshold switch memristor according to the specific application scenario, thereby optimizing the overall fusion effect.

[0084] In summary, the image fusion method based on a multi-band responsive artificial visual spiking neuron array provided by this invention can effectively process image data from different bands and fuse them through a spiking neural network to generate a high-resolution, high-contrast composite image with excellent detail preservation. This reduces the production cost and footprint of artificial visual spiking neuron circuits, laying a solid foundation for building integrated and low-power artificial vision systems. This method has great potential for application in neuromorphic network computing and artificial intelligence hardware.

[0085] Based on the same inventive concept, the second aspect of the present invention also provides an image fusion device based on a multi-band response artificial visual pulse neuron array. Figure 9 , Figure 9 : is a structural block diagram of an image fusion device based on a multi-band response artificial visual pulse neuron array provided by an embodiment of the present invention, the device comprising:

[0086] An image preprocessing module, configured with an artificial visual pulse neuron array, is used to perform image preprocessing on a plurality of optical images of different wavelengths, thereby converting the brightness values ​​of different pixels into pulse signals, thereby obtaining a plurality of corresponding sequence pulse signals. The artificial visual pulse neuron array includes a plurality of artificial visual pulse neurons, each of which includes a photoelectric sensor and a threshold switching memristor. The photoelectric sensor is used to sense optical signals of multiple wavelengths and convert the optical signals into current signals. The memristor is used to encode the current signals into pulse signals.

[0087] The image fusion module is equipped with an artificial visual pulse neural network, which is used to fuse several groups of sequential pulse signals by combining the response characteristics of different bands to obtain a fused pulse image;

[0088] The image decoding module is used to decode the fused pulse image into a pixel image to obtain the final fused image.

[0089] The image fusion device based on the multi-band response artificial visual pulse neuron array provided by the present invention can realize the above-mentioned image fusion method based on the multi-band response artificial visual pulse neuron array. All embodiments of the above-mentioned method are applicable to the device and can achieve the same or similar beneficial effects.

[0090] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0091] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. An image fusion method based on a multi-band response artificial visual pulse neuron array, characterized in that: include: Inputting several groups of optical images of different wavelengths into an artificial visual spiking neuron array and performing image preprocessing to convert the brightness values ​​of different pixels into pulse signals, thereby obtaining several groups of corresponding sequence pulse signals; wherein the artificial visual spiking neuron array includes multiple artificial visual spiking neurons; the artificial visual spiking neurons include a photosensor and a threshold switching memristor; the photosensor is used to sense optical signals of multiple wavelengths and convert the optical signals into current signals; the memristor is used to encode the current signals into pulse signals; Inputting the plurality of groups of sequence pulse signals into an artificial visual pulse neural network, performing fusion processing based on response characteristics of different bands, and obtaining a fused pulse image; The fused pulse image is decoded into a pixel image to obtain a final fused image.

2. The image fusion method based on multi-band response artificial visual pulse neuron array according to claim 1, characterized in that: The specific structure of the artificial visual pulse neuron includes a silicon substrate, a silicon dioxide dielectric layer, a photosensitive layer, a source and drain electrode, a bottom electrode layer, an intermediate layer and a top electrode layer; wherein, The silicon dioxide dielectric layer is located on the silicon substrate; The photosensitive layer is located on the upper left surface of the silicon dioxide dielectric layer; The source and drain electrodes are located on both sides of the photosensitive layer and extend to the lower left or lower right onto the silicon dioxide dielectric layer to form a photosensor on the left side of the entire neuron structure. The silicon substrate serves as both a supporting substrate and a gate for the photosensor. The photosensitive layer has different absorption capabilities for light of different wavelengths, so that the photosensor has multi-band sensing capabilities. The bottom electrode layer is located on the upper right surface of the silicon dioxide dielectric layer; the middle layer completely covers the bottom electrode layer and extends to both sides of the silicon dioxide dielectric layer in a step-like manner; the top electrode layer completely covers the middle layer and extends to both sides of the silicon dioxide dielectric layer in a step-like manner to form a threshold switching memristor on the right side of the entire neuron structure.

3. The image fusion method based on multi-band response artificial visual pulse neuron array according to claim 2, characterized in that: The channel resistance of the photosensor is between the high resistance state and the low resistance state of the threshold switch memristor, so that the artificial visual pulse neuron can encode the output pulse signal.

4. The image fusion method based on multi-band response artificial visual pulse neuron array according to claim 2, characterized in that: The photosensitive layer is realized by a combination of several materials capable of sensing different wavelengths, or by a single material capable of sensing different wavelengths.

5. The image fusion method based on multi-band response artificial visual pulse neuron array according to claim 4 is characterized in that: The photosensitive layer includes a semiconductor layer and an organic layer; the semiconductor layer is located on the upper left surface of the silicon dioxide dielectric layer; the organic layer is located in the middle of the upper surface of the semiconductor layer; wherein, The material of the semiconductor layer is an indium oxide thin film with a thickness of 7-10 nm; the material of the organic layer is a PY-IT thin film with a thickness of 80-120 nm.

6. The image fusion method based on multi-band response artificial visual pulse neuron array according to claim 4 is characterized in that: The photosensitive layer includes a semiconductor layer, the material of the semiconductor layer is an indium selenide thin film with a thickness of 10-50 nm.

7. The image fusion method based on multi-band response artificial visual pulse neuron array according to claim 2, characterized in that: The thickness of the silicon dioxide dielectric layer is 80-150nm; the thickness of the source and drain electrodes is 40-100nm; the bottom electrode layer and the top electrode layer both use metal electrodes composed of titanium / platinum; the middle layer uses niobium oxide material with a thickness of 30-45nm.

8. The image fusion method based on multi-band response artificial visual pulse neuron array according to claim 1 is characterized in that: The artificial visual pulse neural network adopts a multi-layer pulse neuron structure, including an input layer, a hidden layer and an output layer; and the neurons in each layer transmit information through pulses, and the connections between layers complete information transmission through adjustable weights.

9. The image fusion method based on multi-band response artificial visual pulse neuron array according to claim 1, characterized in that: After obtaining the fused pulse image, it also includes: The fused pulse image is subjected to denoising and contrast enhancement operations.

10. An image fusion device based on a multi-band response artificial visual pulse neuron array, used to implement the method according to any one of claims 1 to 9, characterized in that: The device includes: An image preprocessing module, configured with an artificial visual pulse neuron array, is used to perform image preprocessing on a plurality of optical images of different wavelengths to convert the brightness values ​​of different pixels into pulse signals, thereby obtaining a plurality of corresponding sequence pulse signals; wherein the artificial visual pulse neuron array includes a plurality of artificial visual pulse neurons; the artificial visual pulse neurons include a photosensor and a threshold switching memristor; the photosensor is used to sense optical signals of multiple wavelengths and convert the optical signals into current signals; the memristor is used to encode the current signals into pulse signals; An image fusion module is configured with an artificial visual pulse neural network, which is used to fuse the plurality of groups of sequential pulse signals by combining response characteristics of different bands to obtain a fused pulse image; The image decoding module is used to decode the fused pulse image into a pixel image to obtain a final fused image.

Images