A fluorescent photonic neural network, a construction method thereof and an image recognition system

Abstract

The present application relates to the technical field of photonic neural network, and discloses a fluorescent photonic neural network, a construction method thereof and an image recognition system.The construction method comprises the following steps: providing a single-layer photochromic spiropyran film; using a neural network based on a Keras framework to perform simulation training to obtain a weight matrix of a to-be-identified task; writing the weight matrix of the to-be-identified task into the single-layer photochromic spiropyran film, so that the concentration distribution of cyanine on the single-layer photochromic spiropyran film conforms to the weight matrix, and the fluorescent photonic neural network is obtained; and the fluorescent photonic neural network uses the spontaneous fluorescence of the spiropyran film as the signal output of the to-be-identified task.The present application can realize the storage of the computer fitting weight by the spiropyran film by using the mutual conversion of spiropyran and cyanine under the action of ultraviolet light and visible light, and can automatically sense ultraviolet signals and visible light signals.

Description

Technical Field

[0001] The present invention relates to the technical field of photonic neural networks, and in particular to a fluorescent photonic neural network, a construction method thereof, and an image recognition system. Background Art

[0002] Traditional image sensing, recognition, and display processes inevitably require the participation of various electronic components and computers. Due to the limitations of Joule's law and the computer's von Neumann architecture, they inevitably suffer from many problems, such as slow computing speed, high energy consumption, and high heat generation.

[0003] Various existing photonic neural networks based on the photoelectric effect cannot completely get rid of electric current. Either electrical energy input is required to maintain the operation of the neural network, or a photodetector is required to convert the optical signal into an electrical signal in order to realize the computing function of the neural network. Therefore, it is impossible to fundamentally avoid the influence of electric current. For photonic neural networks based on phase change materials, there are problems such as high transparency requirements for phase change materials, severe attenuation of optical signals resulting in only a few layers of neural networks, difficulty in converting crystalline and amorphous states, and single weight values ​​(for example, crystalline and amorphous states can only correspond to two weight values ​​of 0 and 1, respectively). In addition, since existing photonic neural networks cannot emit light on their own, they can only realize a single computing and recognition function, and require the help of a display to display the calculation results, that is, they cannot realize computing and display integration (computing-display integration).

[0004] Therefore, the existing technology needs to be further improved and enhanced. Summary of the Invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention aims to provide a fluorescent photonic neural network and its construction method, as well as an image recognition system, in order to solve the problem that the existing photonic neural network cannot emit light autonomously and cannot achieve integrated computing and display.

[0006] The above-mentioned object of the present invention is achieved through the following technical solution: a method for constructing a fluorescent photonic neural network, which includes the following steps:

[0007] Providing a single-layer photochromic spiropyran film;

[0008] Use the neural network based on the Keras framework for simulation training to obtain the weight matrix of the task to be identified;

[0009] The weight matrix of the task to be identified is written into the single-layer photochromic spiropyran film so that the anthocyanin concentration distribution on the single-layer photochromic spiropyran film conforms to the weight matrix, thereby obtaining the fluorescent photonic neural network; the fluorescent photonic neural network uses the spontaneous fluorescence of the spiropyran film as the signal output of the task to be identified.

[0010] The following is a preferred technical solution of the present application, but not as a limitation of the technical solutions provided by the present application, through the following preferred technical solution, the purpose and beneficial effect of the present application can be better achieved and realized.

[0011] As a preferred technical solution, the preparation method of the single-layer photochromic spiropyran film comprises:

[0012] A certain concentration of 1-(2-hydroxyethyl)-3,3-dimethylindoline-6'-nitrobenzospirpyran is dissolved in methyl methacrylate to obtain a spiropyran mixed solution;

[0013] The spiropyran mixed solution is coated into a film and dried to obtain the single-layer photochromic spiropyran film.

[0014] As a preferred technical solution, the fluorescent photon neural network construction method comprises:

[0015] In the computer, a neural network based on Keras framework is used as a training simulation network, and the weight of the training simulation network is limited to a non-negative value;

[0016] The training data is used to train the training simulation network, and the training data is forward propagated through the training simulation network to calculate the output result;

[0017] The difference between the output result and the target value is evaluated based on a loss function, and the weight is updated by an optimizer until the weight obtained is within the range [0, 1], when the weight is 0, the anthocyanin concentration is 0, and when the weight is 1, the anthocyanin concentration is the highest.

[0018] As a preferred technical solution, the fluorescent photon neural network construction method comprises:

[0019] The weight signal of the task to be identified is simulated by using an ultraviolet projection device, and the weight signal is segmented by a microlens array in the form of spatial light, and the segmented signal is projected on the single-layer photochromic spiropyran film.

[0020] As a preferred technical solution, the fluorescent photon neural network construction method comprises:

[0021] Secondly, a fluorescent photon neural network is provided, wherein the fluorescent photon neural network is constructed by the above-mentioned construction method.

[0022] In a third aspect, an image recognition system, comprising:

[0023] a fluorescent photonic neural network, the fluorescent photonic neural network being the fluorescent photonic neural network of the second aspect;

[0024] an ultraviolet projection device for projecting an image to be recognized onto a single layer of a photochromic spiropyran film in the fluorescent photonic neural network;

[0025] a microlens array for dividing the signal emitted by the ultraviolet projection device;

[0026] a camera for capturing red fluorescence emitted by the single layer of the photochromic spiropyran film;

[0027] The fluorescent photonic neural network, ultraviolet projection device, and microlens array are located on the light path of the projection device.

[0028] As a preferred technical solution, the image recognition system further comprises a visible light source or a laser light source with an emission wavelength of 532 nm, which is used to irradiate the single layer of the photochromic spiropyran film so that the concentration distribution conforms to the cyanine disappearance of the weight matrix.

[0029] As a preferred technical solution, the wavelength of the red fluorescence is 650 nm; and the filter is a 650 nm wavelength filter.

[0030] As a preferred technical solution, the image recognition system further comprises a convex lens for converging the red fluorescence.

[0031] Beneficial effects: Compared with the prior art, the present application first proposes a fluorescent photonic neural network, and basically realizes the integration of sensing, storage, calculation, and display. By utilizing the mutual transformation of spiropyran and cyanine under the action of ultraviolet and visible light, the spiropyran film can store the computer fitting weight, and can automatically sense ultraviolet and visible light signals. At the same time, the fluorescence effect is a photoluminescence effect, and the visual calculation result can be obtained without the aid of a display, which is not possessed by other types of photonic neural networks.

[0032] Secondly, the present application is based on materials rather than devices, so that the high sensitivity, strong stability, and strong plasticity of the spiropyran material can be utilized. On the one hand, the unit volume with sensing, storage, calculation, and display functions can be extremely small and the resolution can be extremely high. On the other hand, compared with the photonic neural network based on photoelectric effect, passive calculation can be realized without any additional energy.

[0033] Finally, it is worth noting that the present application is completely free of current, truly realizing all-optical neural network computing, thereby fundamentally solving the problems of high energy consumption, heating, bandwidth, etc. of traditional image recognition modes and photonic neural networks based on photoelectric effect. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is a fluorescent photonic neural network construction method flowchart provided by the present application;

[0035] Figure 2 is the photochromic situation of the spiropyran film under different UV powers;

[0036] Figure 3 is the photo-fading situation of the spiropyran film under different powers of 532nm;

[0037] Figure 4 is the computer fitting weight writing into the spiropyran film;

[0038] Figure 5 is a fingerprint identification system based on fluorescent light. DETAILED DESCRIPTION

[0039] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely below in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, but not all the embodiments. The description of the at least one exemplary embodiment is actually only illustrative, but not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0041] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and furthermore, it should be understood that when the terms "comprise" and / or "include" are used in the specification, there is a feature, step, operation, device, component and / or combination thereof.

[0042] Unless otherwise specifically stated, the relative arrangement of the parts and steps, numerical expressions and numerical values ​​described in these embodiments do not limit the scope of the present invention. At the same time, it should be clear that, for ease of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship. The techniques, methods and equipment known to ordinary technicians in the relevant fields may not be discussed in detail, but where appropriate, the techniques, methods and equipment should be considered as part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as merely exemplary and not as a limitation. Therefore, other examples of the exemplary embodiments may have different values. It should be noted that similar numbers and letters represent similar items in the following figures, so once an item is defined in one figure, it does not need to be further discussed in subsequent figures.

[0043] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used herein to describe the spatial positional relationship of a device or feature to other devices or features as shown in the figures. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figures. For example, if the device in the drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below their position devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.

[0044] The present invention provides a method for constructing a fluorescent photonic neural network, such as Figure 1 As shown, the following steps are included:

[0045] S10. Providing a single-layer photochromic spiropyran film.

[0046] Specifically, spiropyran is a kind of nitrogen-containing heterocyclic compound, and some spiropyran derivatives have optical properties of photochromism and fluorescence effect. The absorption spectrum change of spiropyran is caused by the isomerization of the molecule under ultraviolet excitation, so as a weight storage material, it has greater advantages in writing speed and spatial resolution accuracy compared with other photochromic materials. After the isomerization of spiropyran into cyanine, it emits fluorescence with good monochromaticity and high intensity, which can be used as a high-sensitivity and stable signal source. Since spiropyran can complete the reverse process of molecular isomerization under the influence of visible light and thermal dynamics, it can also realize the erasing function as a storage material. Therefore, as a storage device, spiropyran can realize high-speed writing and erasing and stable reading by using the different fluorescence efficiencies of the material before and after photochromism.

[0047] Exemplarily, synthesis of a spiropyran film:

[0048] 10wt.% of 1-(2-hydroxyethyl)-3,3-dimethylindolin-6'-nitrobenzospirpyran (SP) was fully dissolved in methyl methacrylate (MMA), and then the mixed solution was uniformly spread on an optical glass slide, and then placed in a drying oven at a temperature of 70 degrees Celsius for about 3 hours, and finally a photochromic spiropyran film with good uniformity was obtained.

[0049] The photochromic properties of the spiropyran film were measured. The prepared spiropyran mixed solution was injected into a 5mm wide quartz cuvette for solidification, a halogen tungsten deuterium lamp was used as the incident light source (350-1100nm), and a Perkin Elmer Lambda 750 spectrophotometer was used to obtain the absorption spectrum. A laser in-situ measurement device with a wavelength of 532nm and a power of 100uw was used to measure the photochromic speed of the 10wt.% spiropyran film under different ultraviolet intensities (see Figure 2 ) and the fading speed under different visible light intensities (taking 532nm wavelength light as an example) (see Figure 3 ).

[0050] S20, a neural network based on a Keras framework is used for simulation training to obtain a weight matrix of a to-be-identified task.

[0051] Specifically, Keras is a high-level neural network API based on Python. It can run on underlying frameworks such as TensorFlow and Theano, has high modularity and ease of use, and can quickly build and experiment with various neural network models. The model in Keras is composed of layers. There are two main types of models: sequential models (Sequential) and functional API models. The present application can select a sequential model.

[0052] Exemplarily, training data and test data are provided, and in a computer, a neural network based on a Keras framework is used for simulation training. The weights of the network are limited to non-negative values and are optimized through a back propagation algorithm. The training data is forward propagated through the network, the output result is calculated, and the difference between the predicted value and the target value is evaluated based on a loss function. Then, the weights are updated by an optimizer such as Adam or SGD, gradually approaching the optimal solution. The final trained weights are in the range [0, 1], and the size of each weight represents the anthocyanin concentration in the simulation. After training, the obtained weights are mapped to the anthocyanin concentration in the actual physical system. For example, when the weight is 0, the anthocyanin concentration is set to 0; when the weight is 1, the concentration is set to the highest value.

[0053] By precisely controlling the anthocyanin concentration distribution, the physical representation of the network weights is realized. In actual operation, the fluorescence intensity of each pixel point is determined by the product of its corresponding anthocyanin concentration and excitation light intensity. Since the fluorescence coefficient of anthocyanin is a constant value, its fluorescence signal is linearly related to the anthocyanin concentration. By adjusting the intensity of the excitation light, the inference process of the neural network can be completed by measuring the output fluorescence distribution. The abstract weights trained in the computer are mapped to the actual physical system, and the optical properties are used for neural network calculation.

[0054] S30, write the weight matrix of the to-be-identified task to the single-layer photochromic spiropyran film, so that the anthocyanin concentration distribution on the single-layer photochromic spiropyran film conforms to the weight matrix, and obtain the fluorescent photonic neural network; the fluorescent photonic neural network uses the spontaneous fluorescence of the spiropyran film as the signal output of the to-be-identified task.

[0055] Specifically, the weight map trained by the computer according to the image data set is projected onto the spiropyran film by a UV projector. At this time, the spiropyran film will produce corresponding anthocyanin according to the pattern projected by the UV projector, and the concentration of the anthocyanin corresponds to the weight in the all-optical neural network. This step completes the migration of the weight trained by the computer to the weight of the all-optical neural network.

[0056] Based on the same inventive concept, the application also provides a fluorescent photonic neural network obtained by using the above construction method. The specific construction method has been described in detail above, and will not be repeated here.

[0057] Based on the same inventive concept, the application also provides an image recognition system, which comprises a DLP optical projector (DLP4710), a microlens array (ML-S1000-F35), a spiropyran film with weight writing completed, a Gige color camera, a convex lens, and a 650nm wavelength filter, etc. The above components are fixed on an optical platform, and since the used light is all spatial light, no waveguide such as optical fiber is needed for connection, and only the focal length alignment light path is needed for use. As shown in Figure 4 .

[0058] The image recognition system is used for fingerprint recognition, and a full-optical neural network prototype machine for the fingerprint recognition system is built, which specifically comprises a DLP optical projector (DLP4710), a microlens array (ML-S1000-F35), a spiropyran film with weight writing completed, a Gige color camera, a convex lens, and a 650nm wavelength filter.

[0059] 8wt.% of 1-(2-hydroxyethyl)-3,3-dimethylindolin-6'-nitrobenzospiropyran (SP) is fully dissolved in methyl methacrylate (MMA), the mixed solution is uniformly spread on an optical glass, and then placed in a drying box for heating at a temperature of 70 degrees Celsius for 2.5 hours, and finally a good uniformity photochromic spiropyran film is obtained.

[0060] The weight writing of the spiropyran film is performed, the DLP optical projector (DLP4710) is used to simulate the fingerprint signal, the simulated fingerprint weight signal is divided by the microlens array (ML-S1000-F35) in the form of spatial light, and the weight writing is completed by projecting on the spiropyran film. The fingerprint pattern to be recognized is projected onto the spiropyran film with weight written by the UV projector, at this time the fingerprint pattern will overlap with the weight pattern, and the final obtained fluorescent pattern is the result of linear operation output. The 650nm red fluorescent image signal output on the film is restored by a zoom lens, filtered by a 650nm wavelength filter, and finally collected by a Gige color camera to complete the recognition of the fingerprint pattern. The cyanine in the spiropyran film is quickly erased by using a 532nm wavelength laser, and thus the recognition of a fingerprint is completed. It should be noted that the fingerprint is used as an example here, and common patterns such as animals and plants can also be recognized, as long as corresponding training data is used for training.

[0061] It should be understood that the application of the application is not limited to the above examples, and those of ordinary skill in the art can make improvements or changes according to the above description, and all these improvements and changes shall belong to the protection scope of the appended claims of the application.

Claims

1. A method for constructing a fluorescent photonic neural network, characterized in that: The steps include: Providing a single-layer photochromic spiropyran film; Use the neural network based on the Keras framework for simulation training to obtain the weight matrix of the task to be identified; Writing the weight matrix of the task to be identified into the single-layer photochromic spiropyran film so that the anthocyanin concentration distribution on the single-layer photochromic spiropyran film conforms to the weight matrix, thereby obtaining the fluorescence-type photonic neural network; the fluorescence-type photonic neural network uses the spontaneous fluorescence of the spiropyran film as the signal output of the task to be identified; The neural network based on the Keras framework is used for simulation training, including: In a computer, a neural network based on the Keras framework is used as a training simulation network, and the weights of the training simulation network are restricted to non-negative values; The training simulation network is trained using training data, the training data is forward propagated through the training simulation network, and an output result is calculated; Evaluate the difference between the output result and the target value based on the loss function, and update the weight through the optimizer until the obtained weight is in the range of [0, 1], when the weight is 0, the anthocyanin concentration is 0, and when the weight is 1, the anthocyanin concentration is the highest value; Writing the weight matrix of the task to be identified into the single-layer photochromic spiropyran film specifically includes: An ultraviolet projection device is used to simulate the weight signal of the task to be identified, and the weight signal is segmented in the form of spatial light through a microlens array. The segmented signal is projected onto the single-layer photochromic spiropyran film.

2. The method for constructing a fluorescent photonic neural network according to claim 1, wherein: The preparation method of the single-layer photochromic spiropyran film comprises: A certain concentration of 1-(2-hydroxyethyl)-3,3-dimethylindoline-6'-nitrobenzospiropyran is dissolved in methyl methacrylate to obtain a spiropyran mixed solution; The spiropyran mixed solution is coated to form a film, and then dried to obtain the single-layer photochromic spiropyran film.

3. The method for constructing a fluorescent photonic neural network according to claim 1, wherein: The thickness of the single-layer photochromic spiropyran film is 1-2 mm.

4. A fluorescent photonic neural network, characterized in that: The fluorescent photonic neural network is constructed using the construction method described in any one of claims 1-3.

5. An image recognition system, characterized in that: include: A fluorescent photonic neural network, wherein the fluorescent photonic neural network is the fluorescent photonic neural network according to claim 4; An ultraviolet projection device is used to project the image to be recognized onto a single-layer photochromic spiropyran film in a fluorescent photonic neural network; a microlens array, for segmenting the signal emitted by the ultraviolet projection device; A camera for capturing fluorescence images, used to capture the red fluorescence emitted by a single-layer photochromic spiropyran film; The fluorescent photonic neural network, ultraviolet projection equipment, and microlens array are located on the optical path of the projection equipment.

6. The image recognition system according to claim 5, characterized in that The image recognition system further includes: a visible light source or a laser light source with an emission wavelength of 532 nm, wherein the visible light source or the laser light source is used to irradiate the single-layer photochromic spiropyran film to eliminate anthocyanins whose concentration distribution conforms to the weight matrix.

7. The image recognition system according to claim 5, characterized in that The wavelength of red fluorescence is 650nm; the filter is a 650nm wavelength filter.

8. The image recognition system according to claim 5, characterized in that The image recognition system further includes a convex lens configured to converge the red fluorescence.

Images