Optical neural network training method, device and equipment, medium and optical neural network chip
By adjusting the magnetic field to change the characteristic parameters of the metasurface structure, dynamically adjusting the polarization channel of the optical neural network, solving the problem of poor flexibility of the optical neural network based on the metasurface structure, and achieving flexible adaptation and efficient processing of different tasks.
Patent Information
- Application Number
- CN202411291003.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-14
AI Technical Summary
The optical neural network based on the metasurface structure cannot dynamically regulate the optical waves, resulting in poor flexibility and adaptability when facing different tasks processing needs.
By adjusting the magnetic field under the current magnetic field state to change the characteristic parameters of the metasurface structure, the polarization channel changes of the optical neural network can be realized, and the transmission of optical signals is dynamically adjusted to meet the needs of different tasks. The specific steps include transmitting the optical signal of the training image in the magnetic field state through the metasurface structure, simulating the optical weight parameters, the activation layer simulates the activation function according to the optical weight parameters, output errors to adjust the network parameters until the preset conditions are reached, and changing the magnetic field to obtain Optical neural networks with different polarization channels.
Dynamic adjustment of optical neural networks is realized, which improves its flexibility and adaptability, can meet the needs of many different tasks, and improves the processing capabilities of optical signals.
Smart Images

Figure CN119089959A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of photonic neural networks, and in particular to an optical neural network training method, device, electronic equipment, computer-readable storage medium, and optical neural network chip. Background Art
[0002] Optical neural networks (ONNs) are a new computing architecture that uses photons as information carriers to realize neural network functions. It uses the volatility and interference effects of photons to perform computing tasks. It has the advantages of high speed, low energy consumption, parallel processing and three-dimensional integration. It is widely used in image recognition, big data processing, complex system simulation, optical communication and biomedicine.
[0003] Optical neural networks based on metasurface structures are a type of optical neural network. Metasurface structures have subwavelength size structures and two-dimensional plane characteristics, which can achieve very high integration, that is, they can integrate more neurons in a smaller area, thereby realizing more complex optical neural networks. At present, the metasurface structure in optical neural networks based on metasurface structures is fixed, and it is impossible to dynamically control light waves, so when facing different task processing requirements, the flexibility and adaptability are relatively poor.
[0004] Therefore, how to solve the above technical problems should be the focus of technical personnel in this field. Summary of the invention
[0005] The purpose of the present invention is to provide an optical neural network training method, device, electronic device, computer-readable storage medium, and optical neural network chip. The optical neural network has adjustable characteristics and can meet different task requirements.
[0006] In order to solve the above technical problems, the present invention provides an optical neural network training method, comprising:
[0007] Step S11, under the current magnetic field state, transmitting the optical signal of the training image through the metasurface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are regulated by the magnetic field;
[0008] Step S12, transmitting the optical signal passing through the metasurface structure to the activation layer, and making the activation layer simulate the activation function of the optical neural network according to the optical weight parameter;
[0009] Step S13, determining an output error between an output result corresponding to the training image and an expected output result, wherein the output result is output by the output layer when the optical signal passing through the activation function is transmitted to the output layer;
[0010] Step S14: when the output error does not meet the target output error requirement, adjusting the network parameters according to the output error until a preset end condition is reached; the network parameters include the optical weight parameters;
[0011] Step S15, adjust the magnetic field to change the polarization channel of the optical neural network, and proceed to step S11 to obtain the optical neural network under different polarization channels.
[0012] As an implementable method, under the current magnetic field state, before transmitting the optical signal of the training image through the metasurface structure in the optical neural network and simulating the optical weight parameters of the training image, it also includes:
[0013] Adjusting the size of the training image so that the training image meets the input requirements of the hypersurface structure;
[0014] The optical weight parameters of the metasurface structure and the initial strength of the magnetic field are initialized according to the adjusted size of the training image.
[0015] As an implementable method, transmitting the optical signal passing through the metasurface structure to the activation layer, and causing the activation layer to simulate the activation function of the optical neural network according to the optical weight parameter includes:
[0016] The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the nonlinear activation function of the optical neural network according to the optical weight parameters.
[0017] As an implementable method, transmitting the optical signal passing through the metasurface structure to the nonlinear activation layer, and causing the nonlinear activation layer to simulate the nonlinear activation function of the optical neural network according to the optical weight parameter includes:
[0018] The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the S-shaped growth curve function of the optical neural network according to the optical weight parameters.
[0019] As an implementable method, transmitting the optical signal passing through the metasurface structure to the nonlinear activation layer, and causing the nonlinear activation layer to simulate the nonlinear activation function of the optical neural network according to the optical weight parameter includes:
[0020] The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the linear rectification function of the optical neural network according to the optical weight parameters.
[0021] As an implementable method, adjusting the optical weight parameter according to the output error until a preset end condition is reached includes:
[0022] Step S141: determining a first gradient of the output error with respect to the optical weight parameter of each neuron in the optical neural network; wherein the first gradient represents the sensitivity of the output error with respect to the optical weight parameter;
[0023] Step S142: determining a second gradient of the output error with respect to the bias of each neuron in the optical neural network; wherein the second gradient represents the sensitivity of the output error with respect to the bias;
[0024] Step S143: adjusting the optical weight parameter and the bias of each neuron using an optimization algorithm according to the first gradient and the second gradient to obtain a new optical weight parameter and a new bias; wherein the adjustment direction of the optical weight parameter is opposite to the direction of the first gradient; and the network parameters include the optical weight parameter and the bias;
[0025] Step S144: When the output layer is back-propagated layer by layer to the input layer of the optical neural network, a trained optical neural network is obtained, and the trained optical neural network is used as a new optical neural network and enters step S11 until a preset end condition is reached, wherein the preset end condition includes that the output error meets the target output error requirement or the number of training times of the optical neural network reaches a preset threshold number of times.
[0026] The present invention also provides an optical neural network training device, comprising:
[0027] The first simulation module is used to execute step S11, under the current magnetic field state, transmit the optical signal of the training image through the metasurface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are regulated by the magnetic field;
[0028] A second simulation module is used to execute step S12, transmit the optical signal passing through the metasurface structure to the activation layer, and make the activation layer simulate the activation function of the optical neural network according to the optical weight parameter;
[0029] A determination module, configured to execute step S13, determine an output error between an output result corresponding to the training image and an expected output result, wherein the output result is output by the output layer when the optical signal passing through the activation function is transmitted to the output layer;
[0030] A first adjustment module, configured to execute step S14, when the output error does not meet the target output error requirement, adjusting the optical weight parameter according to the output error until a preset end condition is reached; the network parameter includes the optical weight parameter;
[0031] The adjustment and iteration module is used to execute step S15, adjust the magnetic field to change the polarization channel of the optical neural network, and enter step S11 to obtain the optical neural network under different polarization channels.
[0032] The present invention also provides an electronic device, comprising
[0033] Memory for storing computer programs;
[0034] A processor is used to implement the steps of any one of the above-mentioned optical neural network training methods when executing the computer program.
[0035] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above-mentioned optical neural network training methods are implemented.
[0036] The present invention also provides an optical neural network chip, which is trained using any of the above-mentioned optical neural network training methods, and includes an input layer, a hidden layer, a nonlinear activation layer and an output layer, wherein the hidden layer includes a multi-layer supersurface structure, and the characteristic parameters of the supersurface structure are regulated by a magnetic field.
[0037] An optical neural network training method provided by the present invention includes: step S11, under the current magnetic field state, transmitting the optical signal of the training image through the super surface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the super surface structure are regulated by the magnetic field; step S12, transmitting the optical signal passing through the super surface structure to the activation layer, and making the activation layer simulate the activation function of the optical neural network according to the optical weight parameters; step S13, determining the output error between the output result corresponding to the training image and the expected output result, and the output result is output by the output layer when the optical signal passing through the activation function is transmitted to the output layer; step S14, when the output error does not meet the target output error requirement, adjusting the network parameters according to the output error until the preset end condition is reached; the network parameters include the optical weight parameters; step S15, adjusting the magnetic field to change the polarization channel of the optical neural network, and entering step S11 to obtain the optical neural network under different polarization channels.
[0038] It can be seen that the optical neural network in the present invention is an optical neural network based on a super surface structure. Under a current magnetic field state, the training image is transmitted to the super surface structure, the optical weight parameters of the training image are simulated, and then the optical signal transmitted through the super surface structure is transmitted to the activation layer. The activation layer simulates the activation function of the optical neural network according to the optical weight parameters, and the activation function processes the input optical signal. The optical signal is then transmitted to the output layer, and then the output result corresponding to the training image is output, and the output error is determined according to the output result and the expected output result, and then the network parameters are adjusted according to the output error until the preset end condition is reached, and the training under the current magnetic field is completed. Changing the magnetic field causes the characteristic parameters of the super surface structure to change, thereby changing the polarization channel of the optical neural network, that is, changing the polarization state of the training image, and then training again to obtain optical neural networks with different polarization channels. Therefore, the present invention can change the characteristic parameters of the super surface structure by adjusting the magnetic field, thereby obtaining optical neural networks with different polarization channels, and realizing dynamic adjustment of optical signals. The dynamically adjustable optical neural network of the present invention can meet the needs of a variety of different tasks and has high flexibility and adaptability.
[0039] In addition, the present invention also provides a device, an electronic device, a computer-readable storage medium and an optical neural network chip having the above advantages. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0041] Figure 1 A process of an optical neural network training method provided by an embodiment of the present invention Figure 1 ;
[0042] Figure 2 A process of an optical neural network training method provided by an embodiment of the present invention Figure 2 ;
[0043] Figure 3 A process of an optical neural network training method provided by an embodiment of the present invention Figure 3 ;
[0044] Figure 4 A structural block diagram of an optical neural network training device provided by an embodiment of the present invention;
[0045] Figure 5A structural block diagram of an electronic device provided by an embodiment of the present invention;
[0046] Figure 6 A schematic diagram of a model of an optical neural network provided by an embodiment of the present invention;
[0047] Figure 7 Schematic diagram of the cascaded metasurface diffraction neural network for multiplexing information. DETAILED DESCRIPTION
[0048] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0049] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0050] As mentioned in the background technology section, the metasurface structure in the current optical neural network based on metasurface structure is fixed and cannot dynamically control the light waves, so the flexibility and adaptability are relatively poor when faced with different task processing requirements.
[0051] In view of this, the present invention provides an optical neural network training method, please refer to Figure 1 , the method may include:
[0052] Step S11, under the current magnetic field state, the optical signal of the training image is transmitted through the metasurface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are regulated by the magnetic field.
[0053] The training image refers to the image used for training the optical neural network, which is not specifically limited in this embodiment and can be selected at will.
[0054] The size of the training images in this step matches the size of the hypersurface structure.
[0055] It should be noted that the current magnetic field state is not limited in this embodiment, and the current magnetic field state can be a state without a magnetic field or a state with a magnetic field. Furthermore, when it is a state with a magnetic field, the intensity, direction, etc. of the magnetic field are not limited in this embodiment and can be set by oneself. In the current magnetic field state, the optical neural network corresponds to a polarization channel.
[0056] The optical signal corresponds to the training image, and the optical signal is obtained by conversion from the input layer after the training image is input into the input layer of the optical neural network.
[0057] Optical signals are also light waves. When optical signals are transmitted forward and enter the metasurface structure, the metasurface structure can regulate the phase, amplitude and polarization of the optical signals. The metasurface structure includes multiple nano-antenna units, which have the characteristics of regulating light waves. A nano-antenna unit is regarded as a metasurface structure neuron.
[0058] The process of the input layer encoding the training image into the hypersurface structure neuron includes: a. Image preprocessing: preprocessing the training image, and the preprocessing methods include grayscale, normalization, etc., so that the training image meets the input requirements of the optical neural network; b. Pixel mapping: mapping each pixel of the training image to the hypersurface structure neuron, and each pixel corresponds to a hypersurface structure neuron; c. Amplitude or phase encoding: each hypersurface structure neuron performs phase encoding according to its corresponding pixel value. For example, the pixel value can be converted into the corresponding amplitude or phase value.
[0059] Before the light wave enters the metasurface structure, the training image is converted into the initial light field distribution. After entering the metasurface structure, the diffraction field distribution is calculated using the Fresnel diffraction integral formula. The diffracted light wave is then transmitted into the activation layer.
[0060] The Fresnel diffraction integral formula is:
[0061] (1)
[0062] In the formula, It is the light waves The diffraction field distribution at It is the light waves The initial light field distribution at is the zero-order Hankel function of the first kind, is the wave number of light, The light wave from arrive The distance Is an imaginary unit.
[0063] The characteristic parameters of the metasurface structure include but are not limited to refractive index, absorptivity, birefringence, etc. The current magnetic field is applied to the metasurface structure, and the material of the metasurface structure is a magnetic control material. When the magnetic field changes, the characteristic parameters of the metasurface structure change, thereby changing the regulation of the optical signal.
[0064] The number of layers of the super surface structure is not limited in this embodiment, and can be two layers, three layers, four layers, etc., and each layer of the super surface structure is cascaded.
[0065] Step S12: transmitting the optical signal passing through the metasurface structure to the activation layer, and making the activation layer simulate the activation function of the optical neural network according to the optical weight parameters.
[0066] In the forward transmission direction of the optical signal, the nonlinear activation layer is located in the next layer of the metasurface structure. The nonlinear activation layer can adjust its optical response according to the optical weight parameters of the optical signal output by the metasurface structure, simulate the activation function in the optical neural network, and further transform and process the optical signal.
[0067] It should be noted that the type of activation function is not limited in this embodiment and depends on the situation.
[0068] The activation layer can be an array composed of nanoantennas, which utilizes the local light field enhancement effect of the nanoantennas to produce an enhanced nonlinear response, and controls the local light field enhancement effect of the nanoantennas by adjusting the intensity of the input light to simulate the activation function in the neural network.
[0069] Step S13, determining an output error between an output result corresponding to the training image and an expected output result, and the output result is output by the output layer when the optical signal passing through the activation function is transmitted to the output layer.
[0070] Step S14: When the output error does not meet the target output error requirement, adjust the network parameters according to the output error until the preset end condition is reached; the network parameters include optical weight parameters.
[0071] The preset end condition may be that the output error meets the target output error requirement or the number of training times of the optical neural network reaches a preset number threshold, etc., which is not specifically limited in this embodiment and can be set by oneself.
[0072] It should be noted that in this embodiment, there is no limitation on the target output error requirement, which can be set at will.
[0073] By adjusting the network parameters to achieve the preset end condition, an optical neural network trained under a polarization state is obtained.
[0074] Step S15, adjust the magnetic field to change the polarization channel of the optical neural network, and enter step S11 to obtain the optical neural network under different polarization channels.
[0075] It should be noted that the adjustment of the magnetic field is not limited in this embodiment and depends on the situation. For example, the absence of a magnetic field can be adjusted to a magnetic field, or the presence of a magnetic field can be adjusted to no magnetic field, or the strength and direction of the magnetic field can be changed when there is a magnetic field.
[0076] When the magnetic field changes, the characteristic parameters of the metasurface structure change, thereby changing the control of the phase, amplitude and polarization of the optical signal, realizing dynamic control of the light wave.
[0077] The characteristic parameters of the metasurface structure change, that is, the change between different polarization channels is realized, and then the optical neural network is trained under different polarization channels to obtain the optical neural network corresponding to the new polarization channel. Therefore, after the training of this embodiment, an optical neural network trained under different polarization channels can be obtained. When facing different processing tasks, an optical neural network under a polarization channel can be selected for task processing according to different task requirements, so that the optical neural network has high flexibility and adaptability.
[0078] The trained optical neural network can be used in tasks such as recognition and classification of optical images. Correspondingly, the output results of the optical neural network are recognition results, classification results, etc.
[0079] In this embodiment, the optical neural network is an optical neural network based on a metasurface structure. Under a current magnetic field state, the training image is transmitted to the metasurface structure, the optical weight parameters of the training image are simulated, and then the optical signal transmitted through the metasurface structure is transmitted to the activation layer. The activation layer simulates the activation function of the optical neural network according to the optical weight parameters, and the activation function then outputs the output result corresponding to the training image. The output error is determined according to the output result output by the activation function and the expected output result, and then the network parameters are adjusted according to the output error until the preset end condition is reached, and the training under the current magnetic field is completed. The magnetic field is changed so that the characteristic parameters of the metasurface structure change, thereby changing the polarization channel of the optical neural network, that is, changing the polarization state of the training image, and then training is performed again to obtain optical neural networks with different polarization channels. Therefore, the present invention can change the characteristic parameters of the metasurface structure by adjusting the magnetic field, thereby obtaining optical neural networks with different polarization channels, and realizing dynamic adjustment of optical signals. The dynamically adjustable optical neural network of the present invention can meet the needs of a variety of different tasks and has high flexibility and adaptability.
[0080] Please refer to Figure 2 Based on the above embodiments, in one embodiment of the present invention, the optical neural network training method may include:
[0081] Step S21: adjust the size of the training image so that the training image meets the input requirements of the hypersurface structure.
[0082] When the size of the training image does not match the hypersurface structure, it is necessary to preprocess the size of the input training image so that the size of the training image matches the hypersurface structure.
[0083] When the size of the input training image does not match the size of the hypersurface structure, the following method can be used to handle it.
[0084] (1) Scaling: When the size of the training image is M×N, and the size of the training image is larger than the size of the hypersurface structure, the training image is scaled to match the size of the hypersurface structure. For example, the training image can be scaled proportionally or divided into multiple parts for processing.
[0085] (2) Sampling: The training image is sampled and converted into a number of pixels that matches the size of the metasurface structure. For example, the training image can be downsampled or upsampled.
[0086] (3) Padding: When the size of the training image is M×N and the size of the training image is smaller than the size of the hypersurface structure, the training image is padded so that its size matches the size of the hypersurface structure. For example, blank pixels can be added around the image or a padding algorithm can be used for padding.
[0087] Step S22: Initialize the optical weight parameters of the metasurface structure and the initial intensity of the magnetic field according to the adjusted size of the training image.
[0088] The initialized optical weight parameters and the initial strength of the magnetic field are not limited in this embodiment and can be set by yourself. When the magnetic field is at the initial strength, the polarization channel corresponding to the optical neural network can be made into the first polarization channel, and each time the magnetic field is adjusted, the polarization channel changes into a new polarization channel, which is called the second polarization channel, the third polarization channel, and so on in sequence.
[0089] Step S23: Under the current magnetic field state, the optical signal of the training image is transmitted through the metasurface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are regulated by the magnetic field.
[0090] It is understandable that, when training for the first time, the current magnetic field is the initial magnetic field strength, corresponding to the first polarization channel of the optical neural network. After the magnetic field is adjusted, the polarization channel changes, and the optical neural network is trained again under the corresponding new polarization channel.
[0091] Step S24, transmitting the optical signal passing through the metasurface structure to the activation layer, and making the activation layer simulate the activation function of the optical neural network according to the optical weight parameters.
[0092] In the forward transmission direction of the optical signal, the nonlinear activation layer is located in the next layer of the metasurface structure. The nonlinear activation layer can adjust its optical response according to the optical weight parameters of the optical signal output by the metasurface structure, simulate the activation function in the optical neural network, and further transform and process the optical signal.
[0093] Step S25, determining an output error between an output result corresponding to the training image and an expected output result, and the output result is output by the output layer when the optical signal passing through the activation function is transmitted to the output layer.
[0094] Step S26: Adjust the network parameters according to the output error until the preset end condition is reached.
[0095] Step S27, adjust the magnetic field to change the polarization channel of the optical neural network, and proceed to step S23 to obtain the optical neural network under different polarization channels.
[0096] It should be noted that steps S23 to S27 may refer to the contents of the above embodiment and will not be described in detail here.
[0097] Based on any of the above embodiments, in one embodiment of the present invention, after obtaining the optical neural network under different polarization channels, the following steps may be further included:
[0098] Use training samples to evaluate the performance of the trained optical neural network, determine the evaluation results such as the classification accuracy, training time and energy consumption of the trained optical neural network, and optimize and adjust the optical neural network based on the evaluation results.
[0099] Please refer to Figure 3 Based on any of the above embodiments, in one embodiment of the present invention, the optical neural network training method may include:
[0100] Step S31, under the current magnetic field state, the optical signal of the training image is transmitted through the metasurface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are regulated by the magnetic field.
[0101] Step S32: transmitting the optical signal passing through the metasurface structure to the nonlinear activation layer, and making the nonlinear activation layer simulate the nonlinear activation function of the optical neural network according to the optical weight parameters.
[0102] The nonlinear activation layer may be an array composed of nano-antenna structures having nonlinear optical effects, and the material of the nonlinear activation layer may be a nonlinear optical material, including but not limited to lithium niobate, lithium tantalate, liquid crystal, and the like.
[0103] The local light field enhancement effect of the nanoantenna is used to produce an enhanced nonlinear response, and the local light field enhancement effect of the nanoantenna is controlled by adjusting the intensity of the input light to simulate the nonlinear activation function in the optical neural network.
[0104] In this embodiment, the simulated optical neural network activation function is a nonlinear activation function, which can introduce nonlinear factors into the optical neural network model and enable the optical neural network to arbitrarily approximate the nonlinear function, so that the optical neural network can be applied to many nonlinear models.
[0105] It should be noted that in this embodiment, the type of nonlinear activation function is not limited and can be selected at will.
[0106] As an implementable method, transmitting the optical signal passing through the metasurface structure to the nonlinear activation layer, and making the nonlinear activation layer simulate the nonlinear activation function of the optical neural network according to the optical weight parameter includes:
[0107] The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the S-shaped growth curve function of the optical neural network according to the optical weight parameters.
[0108] The S-shaped growth curve function can be a Sigmoid function, the output of which is in the range of [0,1], and is particularly suitable for models that output probability. Since any probability value is between 0 and 1, the Sigmoid activation function is the best choice.
[0109] As another possible implementation method, transmitting the optical signal passing through the metasurface structure to the nonlinear activation layer, and making the nonlinear activation layer simulate the nonlinear activation function of the optical neural network according to the optical weight parameter includes:
[0110] The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the linear rectification function of the optical neural network according to the optical weight parameters.
[0111] The linear rectification function may be a ReLU (Rectified Linear Unit) function, which has the advantages of simple calculation, fast convergence speed, reduced overfitting, and alleviation of gradient disappearance.
[0112] When the input optical signal intensity is lower than the preset threshold, the nonlinear response is small, similar to the linear part of the ReLU function; when the input optical signal intensity exceeds the preset threshold, the nonlinear response increases significantly, simulating the saturation part of the ReLU function or the saturation characteristics of the Sigmoid function.
[0113] Step S33, determining an output error between an output result corresponding to the training image and an expected output result, and the output result is output by the output layer when the optical signal after the nonlinear activation function is transmitted to the output layer.
[0114] Step S34: When the output error does not meet the target output error requirement, adjust the network parameters according to the output error until a preset end condition is reached.
[0115] Step S35, adjust the magnetic field to change the polarization channel of the optical neural network, and enter step S31 to obtain the optical neural network under different polarization channels.
[0116] It should be noted that steps S31, S33 to S25 may refer to the contents of the above embodiment and will not be described in detail here.
[0117] Based on any of the above embodiments, in one embodiment of the present invention, adjusting the network parameters according to the output error until the preset end condition is reached includes:
[0118] Step S141: Determine a first gradient of the output error with respect to an optical weight parameter of each neuron in the optical neural network; wherein the first gradient represents the sensitivity of the output error with respect to the optical weight parameter.
[0119] Each layer in an optical neural network consists of neurons.
[0120] The first gradient is used to guide the adjustment direction of the optical weight parameters.
[0121] Step S142: Determine a second gradient of the output error with respect to the bias of each neuron in the optical neural network; wherein the second gradient represents the sensitivity of the output error to the bias.
[0122] Step S143: According to the first gradient and the second gradient, the optical weight parameters and the bias of each neuron are adjusted using an optimization algorithm to obtain new optical weight parameters and new bias; wherein the adjustment direction of the optical weight parameters is opposite to the direction of the first gradient; and the network parameters include the optical weight parameters and the bias.
[0123] In this embodiment, the type of optimization algorithm is not limited and can be selected at will. As an implementable method, the optimization algorithm includes but is not limited to any one of the gradient descent method, the momentum method, the adaptive learning rate method, and the regularization method.
[0124] The adjustment direction of the optical weight parameter is opposite to the direction of the first gradient so that the weight is increased to reduce the output error.
[0125] Step S144: When the output layer is back-propagated layer by layer to the input layer of the optical neural network, a trained optical neural network is obtained, and the trained optical neural network is used as a new optical neural network and enters step S11 until a preset end condition is reached, wherein the preset end condition includes that the output error meets the target output error requirement or the number of training times of the optical neural network reaches a preset threshold number of times.
[0126] In this embodiment, the preset number threshold is not limited and can be set by oneself.
[0127] Back propagation is carried out layer by layer, in the order of output layer, activation layer, hidden layer, and input layer. When all weight parameters and biases are adjusted (that is, updated), the optical neural network is trained once. After multiple repetitions, the performance of the optical neural network can be trained to the best under a magnetic field state, thereby improving the accuracy of task processing.
[0128] Based on any of the above embodiments, in one embodiment of the present invention, determining the output error between the output result corresponding to the training image and the expected output result includes:
[0129] The loss function is used to calculate the difference between the output result and the expected output result as the output error.
[0130] As an implementable method, a cross entropy loss function is used to calculate the difference between the output result and the expected output result to obtain an output error.
[0131] The cross entropy loss function can provide good gradient characteristics in classification problems, especially multi-classification problems, which helps the model converge quickly.
[0132] As another possible implementation, a mean square error loss function is used to calculate the difference between the output result and the expected output result to obtain an output error.
[0133] The mean square error loss function can intuitively measure the square of the difference between the output result and the expected output result, which is easy to understand and explain; the calculation is simple and direct, and the speed is fast; the gradient of the mean square error loss function is continuous and smooth, which helps the stable convergence of the gradient descent algorithm.
[0134] The optical neural network training device provided by an embodiment of the present invention is introduced below. The optical neural network training device described below and the optical neural network training method described above can be referenced to each other.
[0135] Figure 4 The structural block diagram of the optical neural network training device provided by the embodiment of the present invention is shown in FIG. Figure 4 The optical neural network training device may include:
[0136] The first simulation module 100 is used to execute step S11, under the current magnetic field state, transmit the optical signal of the training image through the metasurface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are regulated by the magnetic field;
[0137] The second simulation module 200 is used to execute step S12, transmit the optical signal passing through the metasurface structure to the activation layer, and make the activation layer simulate the activation function of the optical neural network according to the optical weight parameter;
[0138] A determination module 300 is used to execute step S13, determine an output error between an output result corresponding to the training image and an expected output result, and the output result is output by the output layer when the optical signal after the activation function is transmitted to the output layer;
[0139] The first adjustment module 400 is used to execute step S14, adjust the network parameters according to the output error until a preset end condition is reached; the network parameters include optical weight parameters;
[0140] The adjustment and iteration module 500 is used to execute step S15, adjust the magnetic field to change the polarization channel of the optical neural network, and enter step S11 to obtain the optical neural network under different polarization channels.
[0141] The optical neural network training device of this embodiment is used to implement the aforementioned optical neural network training method. Therefore, the specific implementation of the optical neural network training device can be seen in the embodiment of the optical neural network training method in the previous text. For example, the first simulation module 100, the second simulation module 200, the determination module 300, the first adjustment module 400, the adjustment and iteration module 500 are respectively used to implement steps S11, S12, S13, S14 and S15 in the aforementioned optical neural network training method. Therefore, its specific implementation can refer to the description of the corresponding embodiments of each part, which will not be repeated here.
[0142] In the optical neural network training device of this embodiment, the training image is transmitted to the super surface structure under a current magnetic field state, the optical weight parameters of the training image are simulated, and then the optical signal transmitted through the super surface structure is transmitted to the activation layer, the activation layer simulates the activation function of the optical neural network according to the optical weight parameters, and the activation function then outputs the output result corresponding to the training image, and the output error is determined according to the output result output by the activation function and the expected output result, and then the optical weight parameters are adjusted according to the output error until the error meets the target output error requirement, and the training under the current magnetic field is completed. The magnetic field is changed so that the characteristic parameters of the super surface structure change, thereby changing the polarization channel of the optical neural network, that is, changing the polarization state of the training image, and then training is performed again to obtain optical neural networks with different polarization channels. Therefore, the present invention can change the characteristic parameters of the super surface structure by adjusting the magnetic field, thereby obtaining optical neural networks with different polarization channels, and realizing dynamic adjustment of optical signals. The dynamically adjustable optical neural network of the present invention can meet the needs of a variety of different tasks and has high flexibility and adaptability.
[0143] As an implementable embodiment, the optical neural network training device may further include:
[0144] A second adjustment module is used to adjust the size of the training image so that the training image meets the input requirements of the hypersurface structure;
[0145] The initialization module is used to initialize the optical weight parameters of the metasurface structure and the initial strength of the magnetic field according to the adjusted size of the training image.
[0146] As an implementable embodiment, the second simulation module 200 is specifically used to transmit the optical signal passing through the metasurface structure to the nonlinear activation layer, and enable the nonlinear activation layer to simulate the nonlinear activation function of the optical neural network according to the optical weight parameters.
[0147] As an implementable embodiment, the second simulation module 200 is specifically used to transmit the optical signal passing through the metasurface structure to the nonlinear activation layer, and enable the nonlinear activation layer to simulate the S-shaped growth curve function of the optical neural network according to the optical weight parameters.
[0148] As an implementable embodiment, the second simulation module 200 is specifically used to transmit the optical signal passing through the metasurface structure to the nonlinear activation layer, and enable the nonlinear activation layer to simulate the linear rectification function of the optical neural network according to the optical weight parameters.
[0149] As an implementation method, the adjustment and iteration module 500 includes:
[0150] A first determining unit, configured to execute step S141: determining a first gradient of the output error with respect to the optical weight parameter of each neuron in the optical neural network; wherein the first gradient represents a sensitivity of the output error with respect to the optical weight parameter;
[0151] A second determination unit, used in step S142: determining a second gradient of the output error with respect to the bias of each neuron in the optical neural network; wherein the second gradient represents the sensitivity of the output error with respect to the bias;
[0152] An adjustment unit, used in step S143: adjusting the optical weight parameter and the bias of each neuron using an optimization algorithm according to the first gradient and the second gradient to obtain a new optical weight parameter and a new bias; wherein the adjustment direction of the optical weight parameter is opposite to the direction of the first gradient; and the network parameters include the optical weight parameter and the bias;
[0153] A repeating unit is used to execute step S144: when the output layer is back-propagated layer by layer to the input layer of the optical neural network, a trained optical neural network is obtained, the trained optical neural network is used as a new optical neural network and step S11 is entered until a preset end condition is reached, wherein the preset end condition includes that the output error meets the target output error requirement or the number of training times of the optical neural network reaches a preset threshold number of times.
[0154] The electronic device provided by an embodiment of the present invention is introduced below. The electronic device described below and the optical neural network training method described above can be referenced to each other.
[0155] Figure 5 The electronic device provided by the embodiment of the present invention includes:
[0156] A memory 11, used for storing computer programs;
[0157] The processor 12 is used to implement the steps of the optical neural network training method of any of the above embodiments when executing a computer program.
[0158] The computer-readable storage medium provided in an embodiment of the present invention is introduced below. The computer-readable storage medium described below and the optical neural network training method described above can be referenced to each other.
[0159] A computer-readable storage medium having a computer program stored thereon, which implements the steps of the optical neural network training method of any of the above embodiments when the computer program is executed by a processor.
[0160] The present invention also provides an optical neural network chip, which is trained using the optical neural network training method of any of the above embodiments.
[0161] Please refer to Figure 6 The optical neural network chip includes an input layer 1, a hidden layer 2, a nonlinear activation layer 3 and an output layer 4, wherein the hidden layer 2 includes a multi-layer metasurface structure, and the characteristic parameters of the metasurface structure are regulated by a magnetic field.
[0162] The input layer 1 is used to convert the received image into an optical signal and map the image information into the intensity or phase of light.
[0163] The number of layers of the super surface structure in the hidden layer 2 is not limited in this embodiment and can be set at will.
[0164] The schematic diagram of the multi-channel information multiplexing cascade metasurface diffraction neural network is as follows Figure 7 As shown, by cascading multiple metasurface structures, a diffractive optical neural network with multiplexed information can be constructed, which can simultaneously process and recognize input images from different polarization channels and transmit the input images to specific detection areas integrating multiple detectors to realize the recognition and classification of different images.
[0165] The metasurface structure is composed of sub-wavelength scattering units with specific shapes. These scattering units will produce a diffraction effect of light and can modulate the phase, amplitude and polarization of the incident light waves, thereby realizing the encoding and processing of image information.
[0166] The material of the metasurface structure is magnetron material. Under the action of the magnetic field, the characteristic parameters of the magnetron material such as the refractive index, absorptivity and birefringence change. Therefore, the phase, amplitude and polarization modulation state of the scattering unit to the incident light wave can be dynamically adjusted.
[0167] When images of multiple polarization channels are input into a metasurface structure composed of an array of magnetic scattering units, each nanoantenna unit (scattering unit) will produce polarization control characteristics under the action of an external magnetic field, enabling the metasurface structure to precisely control light waves in different polarization states.
[0168] The nonlinear activation layer 3 may be an array composed of nano-antenna structures having nonlinear optical effects, and the material of the nonlinear activation layer may be a nonlinear optical material, including but not limited to lithium niobate, lithium tantalate, liquid crystal, etc. The local light field enhancement effect of the nano-antenna is used to generate an enhanced nonlinear response, and the local light field enhancement effect of the nano-antenna is controlled by adjusting the intensity of the input light to simulate the nonlinear activation function in the optical neural network.
[0169] The output layer 4 is used to output the output results.
[0170] The optical neural network in this embodiment is a reconfigurable all-optical neural network based on a metasurface. Compared with traditional electronic neural networks, the all-optical neural network can achieve faster information processing by utilizing the high-speed propagation characteristics of light. This high-speed processing capability is of great significance for processing large-scale, complex data sets and application scenarios with high real-time requirements.
[0171] The optical neural network in this embodiment can realize complex optical transformation and information processing by cascading multiple magnetically controlled metasurface structures, so that the all-optical neural network has multi-tasking capabilities and is suitable for application in multifunctional and efficient information processing systems.
[0172] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0173] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0174] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0175] The optical neural network training method, device, equipment and medium, and optical neural network chip provided by the present invention are introduced in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the scheme of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.
Claims
1. An optical neural network training method, characterized in that: include: Step S11, under the current magnetic field state, transmitting the optical signal of the training image through the metasurface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are regulated by the magnetic field; Step S12, transmitting the optical signal passing through the metasurface structure to the activation layer, and making the activation layer simulate the activation function of the optical neural network according to the optical weight parameter; Step S13, determining an output error between an output result corresponding to the training image and an expected output result, wherein the output result is output by the output layer when the optical signal passing through the activation function is transmitted to the output layer; Step S14: when the output error does not meet the target output error requirement, adjusting the network parameters according to the output error until a preset end condition is reached; the network parameters include the optical weight parameters; Step S15, adjust the magnetic field to change the polarization channel of the optical neural network, and proceed to step S11 to obtain the optical neural network under different polarization channels.
2. The optical neural network training method according to claim 1, characterized in that: Under the current magnetic field state, the optical signal of the training image is transmitted through the metasurface structure in the optical neural network, and before simulating the optical weight parameters of the training image, the method further includes: Adjusting the size of the training image so that the training image meets the input requirements of the hypersurface structure; The optical weight parameters of the metasurface structure and the initial strength of the magnetic field are initialized according to the adjusted size of the training image.
3. The optical neural network training method according to claim 1, characterized in that: Transmitting the optical signal passing through the metasurface structure to the activation layer, and causing the activation layer to simulate the activation function of the optical neural network according to the optical weight parameter comprises: The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the nonlinear activation function of the optical neural network according to the optical weight parameters.
4. The optical neural network training method according to claim 3, characterized in that: Transmitting the optical signal passing through the metasurface structure to the nonlinear activation layer, and causing the nonlinear activation layer to simulate the nonlinear activation function of the optical neural network according to the optical weight parameter comprises: The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the S-shaped growth curve function of the optical neural network according to the optical weight parameters.
5. The optical neural network training method according to claim 3, characterized in that: Transmitting the optical signal passing through the metasurface structure to the nonlinear activation layer, and causing the nonlinear activation layer to simulate the nonlinear activation function of the optical neural network according to the optical weight parameter comprises: The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the linear rectification function of the optical neural network according to the optical weight parameters.
6. The optical neural network training method according to any one of claims 1 to 5, characterized in that: Adjusting the network parameters according to the output error until a preset end condition is reached includes: Step S141: determining a first gradient of the output error with respect to the optical weight parameter of each neuron in the optical neural network; wherein the first gradient represents the sensitivity of the output error with respect to the optical weight parameter; Step S142: determining a second gradient of the output error with respect to the bias of each neuron in the optical neural network; wherein the second gradient represents the sensitivity of the output error with respect to the bias; Step S143: adjusting the optical weight parameter and the bias of each neuron using an optimization algorithm according to the first gradient and the second gradient to obtain a new optical weight parameter and a new bias; wherein the adjustment direction of the optical weight parameter is opposite to the direction of the first gradient; and the network parameters include the optical weight parameter and the bias; Step S144: When the output layer is back-propagated layer by layer to the input layer of the optical neural network, a trained optical neural network is obtained, the trained optical neural network is used as a new optical neural network and enters step S11 until a preset end condition is reached, wherein the preset end condition includes that the output error meets the target output error requirement or the number of training times of the optical neural network reaches a preset threshold number of times.
7. An optical neural network training device, characterized in that: include: The first simulation module is used to execute step S11, under the current magnetic field state, transmit the optical signal of the training image through the metasurface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are regulated by the magnetic field; A second simulation module is used to execute step S12, transmit the optical signal passing through the metasurface structure to the activation layer, and make the activation layer simulate the activation function of the optical neural network according to the optical weight parameter; A determination module, configured to execute step S13, determine an output error between an output result corresponding to the training image and an expected output result, wherein the output result is output by the output layer when the optical signal passing through the activation function is transmitted to the output layer; A first adjustment module, configured to execute step S14, when the output error does not meet the target output error requirement, adjusting the network parameters according to the output error until a preset end condition is reached; the network parameters include the optical weight parameters; The adjustment and iteration module is used to execute step S15, adjust the magnetic field to change the polarization channel of the optical neural network, and enter step S11 to obtain the optical neural network under different polarization channels.
8. An electronic device, characterized in that: include Memory for storing computer programs; A processor, configured to implement the steps of the optical neural network training method according to any one of claims 1 to 6 when executing the computer program.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the optical neural network training method according to any one of claims 1 to 6 are implemented.
10. An optical neural network chip, characterized in that: The optical neural network is trained by the optical neural network training method according to any one of claims 1 to 6, comprising an input layer, a hidden layer, a nonlinear activation layer and an output layer, wherein the hidden layer comprises a multi-layer metasurface structure, and the characteristic parameters of the metasurface structure are regulated by a magnetic field.
Citation Information
Patent Citations
Pluggable diffraction neural network optimization method based on metasurface and task identification device
CN116596050A
Optical neural network module based on phonon polariton regulation and design method
CN117474063A
Metasurface design method based on deep learning and hologram weight partition regulation and control
CN117933042A
Double-target spatial position and image recognition metasurface device and implementation method
CN118484891A
Neural networks for topology optimization of metasurfaces
WO2024006074A1
Cited By
Optical neural network training method and device, medium and program product
CN119623577A
Inter-satellite laser communication and signal scheduling method based on adaptive polarization modulation
CN120223189A
Implementation method, device and system of physical intrinsic neural network and medium
CN120449954A
