A pluggable bionic optical neural network system with self-adaptive perception capability

Abstract

This application relates to the field of biomimetic neurotechnology and proposes a pluggable biomimetic optical neural network system with adaptive sensing capabilities. The system includes a laser source, a pinhole amplifier, a digital micromirror device, a beam splitter prism unit, a photodetector, a pluggable metasurface structure, a spatial light modulator (SLM), a mirror, and a digital camera with a charge-coupled device (CCD) image sensor. This application can adjust the state of the pluggable metasurface structure according to the contrast of the input image light, and adaptively extract high-frequency information of the object based on the contrast of the surrounding environment, thereby improving the accuracy of target recognition tasks. Furthermore, the activation level of neurons in the diffraction layer can be updated in real time based on the initial classification result of the input image light, improving the dynamic adaptive adjustment capability of visual attention, and thus achieving adaptive adjustment to the perceived target.

Description

Technical Field

[0001] This application relates to the field of bionic neurotechnology, and in particular, to a pluggable bionic optical neural network system with adaptive sensing capabilities. Background Technology

[0002] Visual processing, as the core of intelligent systems' perception of the external environment, is crucial in fields such as target detection. However, with increasingly complex processing tasks, traditional static visual perception systems, lacking dynamic adjustment capabilities, struggle to handle complex scenes. Adaptive perception, on the other hand, allows intelligent systems to dynamically adjust visual attention based on environmental changes and continuously learn new information, thereby significantly improving the accuracy and efficiency of visual processing. Currently, using electro-optical visual sensing devices is the conventional method for achieving adaptive perception visual tasks, which can switch sensing performance by adjusting a third parameter, such as voltage, temperature, or pressure. However, due to the inherent limitations of the von Neumann architecture, traditional electrically operated visual sensing devices suffer from high energy consumption and slow processing speed when handling complex visual tasks, making it difficult to meet the demands of real-time and accurate adaptive perception. Optical neural networks offer a new approach to visual processing, directly processing light signals and effectively solving the energy consumption and speed bottlenecks of traditional electronic neural networks by utilizing the high-speed propagation characteristics of light. However, most current all-optical neural networks still lack adaptive perception capabilities when processing complex environmental information. Summary of the Invention

[0003] This application provides a pluggable bionic optical neural network system with adaptive sensing capabilities to solve one or more technical problems existing in the prior art, and at least provide a beneficial option or create conditions.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to one aspect of the embodiments of this application, a pluggable biomimetic optical neural network system with adaptive sensing capability is provided, the system comprising: A laser source, used to output a light spot; A pinhole amplifier is used to amplify the light spot output from the laser source to obtain an amplified light spot; Digital micromirror device, used to load input images; A beam splitter unit is used to impart light to the input image, obtain the input image light after being illuminated, and split the input image light into a first split beam and a second split beam. A photodetector is used to identify and classify the first beam of light to obtain an initial classification result of the input image light; A pluggable metasurface structure is provided for receiving the second beam splitter light and adjusting the state of the pluggable metasurface structure according to the contrast of the input image light so that the pluggable metasurface structure outputs transmitted light corresponding to the second beam splitter light. A spatial light modulator includes a diffraction layer designed according to biomimetic principles. The spatial light modulator is used to adjust the activation level of neurons in the diffraction layer according to the initial classification result, so that the adjusted diffraction layer diffracts the transmitted light to obtain diffracted light. A mirror, used to reflect the diffracted light; A digital camera with a charge-coupled device image sensor is used to receive the diffracted light reflected by the mirror, and to classify the input image based on the diffracted light reflected by the mirror to obtain a final classification result.

[0006] In one embodiment of this application, based on the foregoing scheme, adjusting the state of the pluggable metasurface structure according to the contrast of the input image light includes: If the contrast of the input image light is less than a preset contrast threshold, the state of the pluggable metasurface structure is adjusted to the insertion state to obtain the transmitted light, and the transmitted light is differentiated to extract the edge information of the input image light. If the contrast of the input image light is greater than or equal to the preset contrast threshold, then the state of the pluggable metasurface structure is adjusted to the unplugged state.

[0007] In one embodiment of this application, based on the aforementioned scheme, the diffraction layer is composed of multiple neurons with adjustable weights, and the activation level of the neurons in the diffraction layer is adjusted by adjusting the weights of each neuron.

[0008] In one embodiment of this application, based on the aforementioned scheme, the pluggable metasurface structure is composed of a silicon dioxide base and an aluminum ridge, and the transmittance of the pluggable metasurface structure is adjusted by controlling the width of the aluminum ridge.

[0009] In one embodiment of this application, based on the foregoing scheme, each neuron consists of nine sub-neurons, and the phase information of each sub-neuron is determined by determining the weights of the neuron. The beneficial effects of this application are as follows: This application proposes a pluggable biomimetic optical neural network system with adaptive sensing capabilities. The pluggable metasurface structure can adjust its state according to the contrast of the input image light, and adaptively extract high-frequency information of the target based on the contrast of the surrounding environment, thereby improving the accuracy of target recognition tasks. Furthermore, the activation level of neurons in the diffraction layer can be updated in real time based on the initial classification results of the input image light, improving the dynamic adaptive adjustment capability of visual attention, and thus achieving adaptive adjustment to the perceived target.

[0010] A pluggable metasurface and a diffraction layer form the core architecture of a biomimetic optical neural network system. The pluggable metasurface structure can be inserted into the front end of the architecture, mimicking the simple receptive field characteristics of human visual systems to extract high-frequency spatial edge information of the input image light. The diffraction layer, through a unique biomimetic design, achieves independent decoupling of phase and weight, forming a diffraction neural network together with a photodetector. Compared with existing technologies, this application has the following advantages: ① The pluggable metasurface structure enables flexible function switching, achieving visual contrast adaptability, and can be flexibly adjusted according to the contrast of the surrounding environment corresponding to the input image light; ② Drawing on the human brain's "sparse activation-priority reinforcement" strategy, it achieves dynamic attention control through real-time updates of neuronal activation levels; ③ Direct optical signal computation solves the bottlenecks of high energy consumption and slow processing speed of traditional electrically operated devices, filling the gap in the adaptive perception capabilities of existing all-optical neural networks.

[0011] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1 A structural diagram of the pluggable biomimetic optical neural network architecture with adaptive sensing capabilities provided in this application; Figure 2 A structural diagram of the pluggable metasurface structure provided in this application; Figure 3 A schematic diagram of the structure of the biomimetic diffraction layer provided in this application. Figure 4 Performance comparison chart of pluggable biomimetic optical neural network architecture with adaptive sensing capabilities provided for this project. Detailed Implementation

[0013] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0014] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0015] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller node devices.

[0016] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0017] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0018] The following describes the structure of echo state networks in existing related technologies: like Figure 1 As shown, Figure 1 The schematic diagram of the structure of the pluggable biomimetic optical neural network system with adaptive sensing capability provided in the embodiments of this application includes a laser source 1, a pinhole amplifier 2, a digital micromirror device (DMD) 3, a beam splitter unit (composed of a beam splitter 4 and a polarizing beam splitter 5), a photodetector 6, a pluggable metasurface structure 7, a spatial light modulator (SLM) 8, a mirror 9, and a digital camera (CCD) with a charge-coupled device image sensor 10.

[0019] The system includes: Laser source 1, which is used to output a light spot; The pinhole amplifier 2 is used to amplify the light spot output by the laser source to obtain an amplified light spot; Digital micromirror device 3, which is used to load input images; A beam splitter unit is used to impart light to the input image, obtain the input image light after being illuminated, and split the input image light into a first split beam and a second split beam. A photodetector 6 is used to identify and classify the first beam of light to obtain an initial classification result of the input image light; A pluggable metasurface structure 7 is used to receive the second beam splitting light and adjust the state of the pluggable metasurface structure according to the contrast of the input image light so that the pluggable metasurface structure outputs transmitted light corresponding to the second beam splitting light. A spatial light modulator 8 includes a diffraction layer designed according to biomimetic principles. The spatial light modulator is used to adjust the activation level of neurons in the diffraction layer according to the initial classification result, so that the adjusted diffraction layer diffracts the transmitted light to obtain diffracted light. Reflector 9, which is used to reflect the diffracted light; A digital camera 10 with a charge-coupled device image sensor is used to receive the diffracted light reflected by the mirror, and to classify the input image based on the diffracted light reflected by the mirror to obtain a final classification result.

[0020] The following is the light flow path of the input image: 1. The light spot output from the 532nm laser source 1 is magnified to a diameter of 20mm by the pinhole amplifier 2 to obtain the magnified light spot, and then the magnified light spot is directed towards the digital micromirror device 3 (DMD). 2. The input image (i.e., the environmental image) is loaded onto the DMD for the first modulation, which is to apply light to the input image, thus obtaining the input image light. 3. When the contrast of the input image is low, the pluggable metasurface structure can be inserted to extract the edges of the input image. The light passing through the pluggable metasurface structure (i.e. the second beam split) only carries the edge information of the input image downwards.

[0021] 4. The input image light is split into two beams of equal intensity (the first beam and the second beam) by the beam splitter 4. The first beam is directed to the photodetector, which makes a preliminary judgment to determine the approximate classification of the input image and obtains the initial classification result. Then, the activation level of the neurons in the diffraction layer is dynamically adjusted based on this initial classification result.

[0022] 5. Another beam of light (the second beam of light) is directed onto the diffraction layer on the SLM, which has been adjusted for the activation level of neurons. The phase information corresponding to the weight layer is identified by a digital camera (CCD) 10 with a charge-coupled device image sensor, and the final classification result is output.

[0023] The detailed structure of the pluggable metasurface described in this article is as follows: This metasurface uses SiO2 (silicon dioxide) as a substrate, with a core of a concentric array of aluminum rings with a diameter of 2 mm. The height of the aluminum rings is fixed at 70 nm, the unit cell period is 500 nm, and the width of the aluminum rings can be adjusted between 0 and 500 nm (minimum 50 nm, limited by the fabrication process), enabling the transmission amplitude to gradually increase from the center (0) to the edge (1). Its cylindrical symmetry gives the device 2D isotropic and polarization-independent characteristics, making it suitable for visible light (450-800 nm) and near-infrared bands, meeting the optical response requirements of all-optical edge detection.

[0024] The detailed design of the novel diffraction layer based on biomimetic design described in this article is as follows: First, train a single diffraction layer that can recognize multiple types of objects, that is, a diffraction layer that can correspond to different initial classification results; Add a weight layer and train it on specific classification results to obtain weight layers for different classification results; Each neuron in the original diffraction layer is divided into 9 identical sub-neurons; Combined with the weight layer, when the weight is 1, the phase of the sub-neuron is not changed, and when the weight is 0, the phase of the 3 sub-neurons in the middle row of the 9 sub-neurons is reversed. After processing all neurons, their phases are represented to 0-2. Then, project it onto the SLM.

[0025] like Figure 2 As shown, the pluggable metasurface structure 7 includes a SiO2 base 12 and an aluminum ridge 13.

[0026] like Figure 3 As shown, the novel diffraction layer structure 14 based on biomimetic design includes a neuron structure 15 with a weight of 1, a neuron structure 16 with a weight of 7 / 9, and a neuron structure 17 with a weight of 3 / 9.

[0027] The method for fabricating the pluggable metasurface described in this article is as follows: 1. Substrate preparation: Fused silica wafers are selected as substrates to ensure optical transparency and structural stability.

[0028] 2. Photoresist treatment: Spin-coat PMMA photoresist onto the substrate (2000 rpm, 45 s), bake at 180℃ for 60 s to cure, and then coat with conductive polymer (DisCharge DI) to prevent charge accumulation during electron beam exposure. 3. Patterning and Development: Expose the Al ring pattern using electron beam lithography (50kV), clean the conductive polymer with deionized water, and develop with an IPA / MIBK (3:1) mixture for 60 seconds.

[0029] 4. Metal deposition and stripping: 70 nm thick Al was deposited by evaporation at a rate of 0.5 Å / s, and the photoresist was stripped by an acetone ultrasonic bath, preserving the Al ring structure.

[0030] 5. Finished product characterization: Observe the Al ring morphology, width uniformity, and height (70nm) to ensure that the transmission amplitude gradient meets the design requirements.

[0031] like Figure 4 As shown, Figure 4 The diagram illustrates the performance improvement of the pluggable biomimetic optical neural network architecture with adaptive sensing capabilities provided in the embodiments of this application.

[0032] In summary, the beneficial effects of this application compared with the prior art are as follows: 1. Achieve adaptive visual contrast, with high recognition accuracy in low-contrast environments (such as smog and dust storms). The single-layer architecture achieves a comprehensive recognition rate of 96%, which is more than 40% higher than traditional optical neural networks.

[0033] 2. It has the ability to dynamically regulate visual attention. Through the "sparse activation-priority reinforcement" strategy and improved diffraction architecture, it updates the activation / inhibition state of neurons in real time, achieving a recognition rate of 96%, which is more than 14% higher than traditional technologies.

[0034] 3. The pluggable design allows for flexible switching of functions. Inserting it into the metasurface enables contrast adaptation, while removing it enables attention control, adapting to different scenario needs.

[0035] 4. Inheriting the advantages of optical neural networks, it directly calculates optical signals to solve the bottlenecks of high energy consumption and slow processing of traditional electrical operating devices, while also making up for the shortcomings of existing all-optical network adaptive sensing.

[0036] 5. Bionic design is suitable for complex environments. By drawing on the "filter" mechanism and efficient perception strategies of the human visual system, the applicability of intelligent systems in real and complex scenarios is improved.

[0037] The above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this application and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0038] It should be understood that this application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A pluggable biomimetic optical neural network system with adaptive sensing capabilities, characterized in that, The system includes: A laser source, used to output a light spot; A pinhole amplifier is used to amplify the light spot output from the laser source to obtain an amplified light spot; Digital micromirror device, used to load input images; A beam splitter unit is used to impart light to the input image, obtain the input image light after being illuminated, and split the input image light into a first split beam and a second split beam. A photodetector is used to identify and classify the first beam of light to obtain an initial classification result of the input image light; A pluggable metasurface structure is provided for receiving the second beam splitter light and adjusting the state of the pluggable metasurface structure according to the contrast of the input image light so that the pluggable metasurface structure outputs transmitted light corresponding to the second beam splitter light. A spatial light modulator includes a diffraction layer designed according to biomimetic principles. The spatial light modulator is used to adjust the activation level of neurons in the diffraction layer according to the initial classification result, so that the adjusted diffraction layer diffracts the transmitted light to obtain diffracted light. A mirror, used to reflect the diffracted light; A digital camera with a charge-coupled device image sensor is used to receive the diffracted light reflected by the mirror, and to classify the input image based on the diffracted light reflected by the mirror to obtain a final classification result.

2. The pluggable biomimetic optical neural network system with adaptive sensing capability according to claim 1, characterized in that, Adjusting the state of the pluggable metasurface structure based on the contrast of the input image light includes: If the contrast of the input image light is less than a preset contrast threshold, the state of the pluggable metasurface structure is adjusted to the insertion state to obtain the transmitted light, and the transmitted light is differentiated to extract the edge information of the input image light. If the contrast of the input image light is greater than or equal to the preset contrast threshold, then the state of the pluggable metasurface structure is adjusted to the unplugged state.

3. The pluggable biomimetic optical neural network system with adaptive sensing capability according to claim 1, characterized in that, The diffraction layer is composed of multiple neurons with adjustable weights. The activation level of the neurons in the diffraction layer can be adjusted by adjusting the weights of each neuron.

4. The pluggable biomimetic optical neural network system with adaptive sensing capability according to claim 1, characterized in that, The pluggable metasurface structure consists of a silicon dioxide base and an aluminum ridge. The transmittance of the pluggable metasurface structure is adjusted by controlling the width of the aluminum ridge.

5. The pluggable biomimetic optical neural network system with adaptive sensing capability according to claim 3, characterized in that, Each neuron consists of nine sub-neurons, and the phase information of each sub-neuron is determined by determining the weights of the neuron.

Images