Fluorescence type photon neural network, construction method thereof and image recognition system

Abstract

The invention relates to the technical field of photon neural networks, and discloses a fluorescence type photon neural network, a construction method thereof and an image recognition system. The construction method comprises the following steps: providing a single-layer photochromic spiropyrane film; performing simulation training by adopting a neural network based on a Keras framework to obtain a weight matrix of the to-be-identified task; writing the weight matrix of the to-be-identified task into the single-layer photochromic spiropyrane film, so that the anthocyanin concentration distribution on the single-layer photochromic spiropyrane film conforms to the weight matrix, and obtaining the fluorescent photon neural network; and the fluorescence type photon neural network takes spontaneous fluorescence of the spiropyrane film as signal output of a task to be identified. According to the method, the spiropyrane and the anthocyanin are mutually converted under the action of the ultraviolet light and the visible light, so that the spiropyrane thin film can store the fitting weight of a computer, and the ultraviolet signal and the visible light signal can be automatically sensed.

Description

Technical Field

[0001] The present invention relates to the technical field of photon neural networks, and in particular to a fluorescent photon neural network and a construction method thereof, and an image recognition system. Background Art

[0002] The traditional image sensing, recognition and display process inevitably requires the participation of various electronic components and computers. Due to the limitations of Joule's law and the von Neumann structure of computers, there are bound to be many problems such as slow computing speed, high energy consumption, and high heat generation.

[0003] Various existing photonic neural networks based on the photoelectric effect cannot completely get rid of electric current. Either electrical energy input is required to maintain the operation of the neural network, or a photodetector is required to convert the optical signal into an electrical signal to realize the computing function of the neural network. Therefore, it is impossible to fundamentally avoid the influence of electric current. For photonic neural networks based on phase change materials, there are problems such as high transparency requirements for phase change materials, severe attenuation of optical signals resulting in only a few layers of neural networks, difficulty in converting crystalline and amorphous states, and single weight values ​​(for example, crystalline and amorphous states can only correspond to two weight values ​​of 0 and 1, respectively). In addition, since existing photonic neural networks cannot emit light autonomously, they can only realize a single computing and recognition function, and require the help of a display to display the calculation results, that is, they cannot realize computing-display integration (computing-display integration).

[0004] Therefore, the prior art needs to be further improved and enhanced. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a fluorescent photonic neural network and its construction method and image recognition system, aiming to solve the problem that the existing photonic neural network cannot emit light autonomously and cannot realize integrated computing and display.

[0006] The above object of the present invention is achieved through the following technical solution: a method for constructing a fluorescent photonic neural network, which comprises the following steps:

[0007] Providing a single-layer photochromic spiropyran film;

[0008] A neural network based on the Keras framework is used for simulation training to obtain the weight matrix of the task to be identified;

[0009] The weight matrix of the task to be identified is written into the single-layer photochromic spiropyran film so that the anthocyanin concentration distribution on the single-layer photochromic spiropyran film conforms to the weight matrix, thereby obtaining the fluorescent photonic neural network; the fluorescent photonic neural network uses the spontaneous fluorescence of the spiropyran film as the signal output of the task to be identified.

[0010] The following are preferred technical solutions of the present invention, but are not intended to limit the technical solutions provided by the present invention. Through the following preferred technical solutions, the objectives and beneficial effects of the present invention can be better achieved and realized.

[0011] As a preferred technical solution, the fluorescent photonic neural network construction method, wherein the preparation method of the single-layer photochromic spiropyran film comprises:

[0012] Dissolving a certain concentration of 1-(2-hydroxyethyl)-3,3-dimethylindoline-6'-nitrobenzospiropyran in methyl methacrylate to obtain a spiropyran mixed solution;

[0013] The spiropyran mixed solution is coated into a film, and then dried to obtain the single-layer photochromic spiropyran film.

[0014] As a preferred technical solution, the fluorescent photonic neural network construction method, wherein a neural network based on the Keras framework is used for simulation training, specifically includes:

[0015] In a computer, a neural network based on a Keras framework is used as a training simulation network, and the weights of the training simulation network are restricted to non-negative values;

[0016] The training simulation network is trained using training data, the training data is forward propagated through the training simulation network, and an output result is calculated;

[0017] The difference between the output result and the target value is evaluated based on the loss function, and the weight is updated by the optimizer until the obtained weight is in the range of [0,1]. When the weight is 0, the anthocyanin concentration is 0, and when the weight is 1, the anthocyanin concentration is the highest value.

[0018] As a preferred technical solution, the fluorescent photonic neural network construction method, wherein the weight matrix of the task to be identified is written into the single-layer photochromic spiropyran film, specifically comprises:

[0019] The weight signal of the task to be identified is simulated by using an ultraviolet projection device, and the weight signal is segmented in the form of spatial light through a microlens array, and the segmented signal is projected onto the single-layer photochromic spiropyran film.

[0020] As a preferred technical solution, in the fluorescent photonic neural network construction method, the thickness of the single-layer photochromic spiropyran film is 1-2 mm.

[0021] In a second aspect, a fluorescent photonic neural network is provided, wherein the fluorescent photonic neural network is constructed using the construction method described above.

[0022] In a third aspect, an image recognition system comprises:

[0023] A fluorescent photonic neural network, wherein the fluorescent photonic neural network is the fluorescent photonic neural network described in the second aspect;

[0024] An ultraviolet projection device is used to project the image to be recognized onto a single-layer photochromic spiropyran film in a fluorescent photonic neural network;

[0025] A microlens array, used for segmenting the signal emitted by the ultraviolet projection device;

[0026] A camera for capturing fluorescence images, used to capture the red fluorescence emitted by a single layer of photochromic spiropyran film;

[0027] The fluorescent photonic neural network, ultraviolet projection equipment, and microlens array are located on the optical path of the projection equipment.

[0028] As a preferred technical solution, the image recognition system, wherein the image recognition system also includes: a visible light source or a laser light source with an emission wavelength of 532nm, the visible light source and the laser light source are used to irradiate the single-layer photochromic spiropyran film to make the anthocyanins whose concentration distribution conforms to the weight matrix disappear.

[0029] As a preferred technical solution, in the image recognition system, the wavelength of red fluorescence is 650nm; and the filter is a 650nm wavelength filter.

[0030] As a preferred technical solution, the image recognition system further comprises: a convex lens, wherein the convex lens is used to converge the red fluorescence.

[0031] Beneficial effects: Compared with the prior art, the present invention takes the lead in proposing a fluorescent photonic neural network, which basically realizes the integration of sensing, storage, calculation and display. By utilizing the mutual conversion of spiropyran and anthocyanin under the action of ultraviolet light and visible light, the spiropyran film can store the computer fitting weights and can automatically sense ultraviolet signals and visible light signals. At the same time, the fluorescence effect is a photoluminescent effect, and the visualized calculation results can be obtained without the aid of a display, which is not available in other types of photonic neural networks.

[0032] Secondly, the present invention is based on materials rather than devices, and therefore can utilize the high sensitivity, strong stability, and strong plasticity of spiropyran materials. On the one hand, the unit with sensing, storage, computing, and display functions can be extremely small and have extremely high resolution. On the other hand, compared with photonic neural networks based on the photoelectric effect, passive computing can be achieved without any external energy.

[0033] Finally, it is worth noting that the present invention completely gets rid of electric current and truly realizes all-optical neural network computing, thereby fundamentally solving the problems of high energy consumption, heat generation, bandwidth, etc. of traditional image recognition modes and photonic neural networks based on the photoelectric effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of a process of constructing a fluorescent photonic neural network provided by the present invention;

[0035] Figure 2 is the photochromic change of spiropyran film under different UV powers;

[0036] Figure 3 The photofading of spiropyran film at different powers at 532nm;

[0037] Figure 4 It is the computer fitting weights written into the spiropyran film;

[0038] Figure 5 It is a fingerprint recognition system based on fluorescence. DETAILED DESCRIPTION

[0039] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0042] Unless otherwise specifically stated, the relative arrangement of the parts and steps described in these embodiments, the numerical expressions and numerical values ​​do not limit the scope of the present invention. At the same time, it should be clear that, for ease of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship. The technology, methods and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technology, methods and equipment should be regarded as part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0043] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below their position devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.

[0044] The present invention provides a method for constructing a fluorescent photonic neural network, such as Figure 1 As shown, the following steps are included:

[0045] S10, providing a single-layer photochromic spiropyran film.

[0046] Specifically, spiropyran is a nitrogen-containing heterocyclic compound, and some spiropyran derivatives have the optical properties of photochromic and fluorescent effects. The change in the absorption spectrum of spiropyran is caused by the isomerization of its molecules under ultraviolet excitation, so as a weight storage material, it has greater advantages than other photochromic materials in terms of writing speed and spatial resolution accuracy. After spiropyran is isomerized into anthocyanin, the fluorescence it emits when excited has good monochromaticity and high intensity, and can be used as a highly sensitive and stable signal source. Because spiropyran will complete the reverse process of molecular isomerization under the influence of visible light and thermodynamics, it can also realize the erasing function as a storage material. Therefore, as a storage device, spiropyran can achieve high-speed writing and erasing and stable reading by utilizing the different fluorescence efficiencies of the material before and after photochromism.

[0047] Exemplarily, the synthesis of spiropyran thin films:

[0048] 10wt.% of 1-(2-hydroxyethyl)-3,3-dimethylindoline-6'-nitrobenzospiropyran (SP) is fully dissolved in methyl methacrylate (MMA), and the mixed solution is evenly spread on an optical glass slide, and then placed in a drying oven and heated at 70 degrees Celsius for about 3 hours, and finally a photochromic spiropyran film with good uniformity can be obtained.

[0049] The photochromic properties of spiropyran film were measured by injecting the prepared spiropyran mixed solution into a 5mm wide quartz cuvette for solidification, using a halogen tungsten deuterium lamp as the incident light source (350-1100nm) and using a Perkin Elmer Lambda 750 UV-Vis spectrophotometer to obtain its absorption spectrum. The photochromic speed of 10wt% spiropyran film under different UV intensities was measured using a laser in-situ measurement device with a wavelength of 532nm and a power of 100uW (see Figure 2 ) and the fading speed under different visible light intensities (taking 532nm wavelength light as an example) (see Figure 3 ).

[0050] S20. Use a neural network based on the Keras framework for simulation training to obtain the weight matrix of the task to be identified.

[0051] Specifically, Keras is a high-level neural network API based on Python. It can run on underlying frameworks such as TensorFlow and Theano, has a high degree of modularity and ease of use, and can quickly build and experiment with various neural network models. The model in Keras is composed of layers. There are two main types of models: sequential models and functional API models. The present invention can choose sequential models.

[0052] Exemplarily, training data and test data are provided, and a neural network based on the Keras framework is used in a computer for simulation training. The weights of the network are restricted to non-negative values ​​and are optimized by a back-propagation algorithm. The training data is forward propagated through the network, the output result is calculated, and the difference between the predicted value and the target value is evaluated based on the loss function. Subsequently, the weights are updated by an optimizer (such as Adam or SGD) to gradually approach the optimal solution. The weights obtained by the final training are in the range [0,1], and the size of each weight represents the anthocyanin concentration in the simulation. After the training is completed, the obtained weights are mapped to the anthocyanin concentration in the actual physical system. For example, when the weight is 0, the anthocyanin concentration is set to 0; when the weight is 1, the concentration is set to the highest value.

[0053] By precisely controlling the distribution of anthocyanin concentration, the physical representation of network weights is achieved. In actual operation, the fluorescence intensity of each pixel is determined by the product of its corresponding anthocyanin concentration and the intensity of the excitation light. Since the fluorescence coefficient of anthocyanin is a fixed value, its fluorescence signal is linearly related to the anthocyanin concentration. By adjusting the intensity of the excitation light and measuring the output fluorescence distribution, the reasoning process of the neural network can be completed. The abstract weights trained in the computer are mapped to the actual physical system, and the optical properties are used for neural network calculations.

[0054] S30, writing the weight matrix of the task to be identified into the single-layer photochromic spiropyran film, so that the anthocyanin concentration distribution on the single-layer photochromic spiropyran film conforms to the weight matrix, and obtaining the fluorescent photonic neural network; the fluorescent photonic neural network uses the spontaneous fluorescence of the spiropyran film as the signal output of the task to be identified.

[0055] Specifically, the weight map obtained by computer training based on the image data set is projected onto the spiropyran film by a UV projector. At this time, the spiropyran film will produce corresponding anthocyanins according to the pattern of UV projection. The concentration of anthocyanins corresponds to the weights in the all-optical neural network. This step completes the migration of the weights trained by the computer to the weights of the all-optical neural network.

[0056] Based on the same inventive concept, the present invention also provides a fluorescent photonic neural network, which is obtained by adopting the above-mentioned construction method. The specific construction method has been introduced in detail above and will not be repeated here.

[0057] Based on the same inventive concept, the present invention also provides an image recognition system, including: DLP optical projector (DLP4710), microlens array (ML-S1000-F35), spiropyran film with weight writing completed, Gige color camera, convex lens and 650nm wavelength filter, etc. The above components are fixed on the optical platform. Because the light used is spatial light, there is no need for waveguides such as optical fibers to connect. It only needs to align the focal length and the optical path before use. Figure 4 shown.

[0058] The image recognition system is used to identify fingerprints, and an all-optical neural network prototype for the fingerprint recognition system is built. Its specific structure includes a DLP optical projector (DLP4710), a microlens array (ML-S1000-F35), a spiropyran film with weight writing, a Gige color camera, a convex lens, and a 650nm wavelength filter.

[0059] 8wt.% of 1-(2-hydroxyethyl)-3,3-dimethylindoline-6'-nitrobenzospiropyran (SP) was fully dissolved in methyl methacrylate (MMA), and the mixed solution was evenly spread on an optical glass slide, and then placed in a drying oven and heated at 70 degrees Celsius for 2.5 hours, and finally a photochromic spiropyran film with good uniformity was obtained.

[0060] The weights are written to the spiropyran film, and the fingerprint signal is simulated by a DLP optical projector (DLP4710). The simulated fingerprint weight signal is segmented in the form of spatial light through a microlens array (ML-S1000-F35), and projected onto the spiropyran film to complete the weight writing. The fingerprint pattern to be identified is projected by a UV projector onto the spiropyran film with weights written. At this time, the fingerprint pattern will overlap with the weight pattern, and the final fluorescent pattern is the result of the linear operation output. The 650nm red fluorescent image signal output on the film is restored by a zoom lens, filtered by a 650nm wavelength filter, and finally collected by a Gige color camera to complete the recognition of the fingerprint pattern. The anthocyanin in the spiropyran film is quickly erased using a 532nm wavelength laser, and the fingerprint recognition is completed. It should be noted that fingerprints are used as an example here, and common graphics such as animals and plants can also be identified, and only the corresponding training data needs to be used for training.

[0061] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for constructing a fluorescent photonic neural network, characterized in that: The steps include: Providing a single-layer photochromic spiropyran film; A neural network based on the Keras framework is used for simulation training to obtain the weight matrix of the task to be identified; The weight matrix of the task to be identified is written into the single-layer photochromic spiropyran film so that the anthocyanin concentration distribution on the single-layer photochromic spiropyran film conforms to the weight matrix, thereby obtaining the fluorescent photonic neural network; the fluorescent photonic neural network uses the spontaneous fluorescence of the spiropyran film as the signal output of the task to be identified.

2. The method for constructing a fluorescent photonic neural network according to claim 1, characterized in that: The preparation method of the single-layer photochromic spiropyran film comprises: Dissolving a certain concentration of 1-(2-hydroxyethyl)-3,3-dimethylindoline-6'-nitrobenzospiropyran in methyl methacrylate to obtain a spiropyran mixed solution; The spiropyran mixed solution is coated into a film, and then dried to obtain the single-layer photochromic spiropyran film.

3. The method for constructing a fluorescent photonic neural network according to claim 1, characterized in that: The neural network based on Keras framework is used for simulation training, including: In a computer, a neural network based on a Keras framework is used as a training simulation network, and the weights of the training simulation network are restricted to non-negative values; The training simulation network is trained using training data, the training data is forward propagated through the training simulation network, and an output result is calculated; The difference between the output result and the target value is evaluated based on the loss function, and the weight is updated by the optimizer until the obtained weight is in the range of [0,1]. When the weight is 0, the anthocyanin concentration is 0, and when the weight is 1, the anthocyanin concentration is the highest value.

4. The method for constructing a fluorescent photonic neural network according to claim 1, characterized in that: Writing the weight matrix of the task to be identified into the single-layer photochromic spiropyran film specifically comprises: The weight signal of the task to be identified is simulated by using an ultraviolet projection device, and the weight signal is segmented in the form of spatial light through a microlens array, and the segmented signal is projected onto the single-layer photochromic spiropyran film.

5. The method for constructing a fluorescent photonic neural network according to claim 1, characterized in that: The thickness of the single-layer photochromic spiropyran film is 1-2 mm.

6. A fluorescent photonic neural network, characterized in that: The fluorescent photonic neural network is constructed using the construction method described in any one of claims 1-5.

7. An image recognition system, characterized in that: include: A fluorescent photonic neural network, wherein the fluorescent photonic neural network is the fluorescent photonic neural network according to claim 6; An ultraviolet projection device is used to project the image to be recognized onto a single-layer photochromic spiropyran film in a fluorescent photonic neural network; A microlens array, used for segmenting the signal emitted by the ultraviolet projection device; A camera for capturing fluorescence images, used to capture the red fluorescence emitted by a single layer of photochromic spiropyran film; The fluorescent photonic neural network, ultraviolet projection equipment, and microlens array are located on the optical path of the projection equipment.

8. The image recognition system according to claim 7, characterized in that: The image recognition system also includes: a visible light source or a laser light source with an emission wavelength of 532nm, and the visible light source and the laser light source are used to irradiate the single-layer photochromic spiropyran film to make the anthocyanins whose concentration distribution conforms to the weight matrix disappear.

9. The image recognition system according to claim 7, characterized in that: The wavelength of red fluorescence is 650nm; the filter is a 650nm wavelength filter.

10. The image recognition system according to claim 7, characterized in that: The image recognition system further includes a convex lens, which is used to converge the red fluorescence.

Images