Optical neural network training method and device, equipment and medium, optical neural network chip

Abstract

The present application relates to the field of photonic neural networks, and discloses an optical neural network training method, device, equipment and medium, and an optical neural network chip, comprising the following steps: S11, transmitting an optical signal of a training image through a metasurface structure of an optical neural network under a current magnetic field state to simulate an optical weight parameter of the training image; the characteristic parameter of the metasurface structure is regulated by the magnetic field; S12, transmitting the optical signal to an activation layer and making the activation layer simulate an activation function of the neural network according to the optical weight parameter; S13, determining an output error between an output result corresponding to the training image and an expected output result; S14, adjusting a network parameter according to the output error; S15, adjusting the magnetic field to change a polarization channel of the neural network, and entering S11 to obtain the neural network under different polarization channels. The present application can dynamically adjust the optical neural network to meet the needs of various different tasks, and improve flexibility and adaptability.

Description

Technical Field

[0001] This invention relates to the field of photonic neural networks, and in particular to an optical neural network training method, apparatus, electronic device, computer-readable storage medium, and optical neural network chip. Background Technology

[0002] Optical neural networks (ONNs) are a novel computing architecture that uses photons as information carriers to realize neural network functions. They utilize the wave properties and interference effects of photons to perform computational tasks and have advantages such as high speed, low power consumption, parallel processing, and three-dimensional integration. They are widely used in fields such as image recognition, big data processing, complex system simulation, optical communication, and biomedicine.

[0003] Optical neural networks based on metasurface structures are a type of optical neural network. Metasurface structures possess subwavelength dimensions and two-dimensional planar characteristics, enabling very high integration density—that is, they can integrate more neurons within a smaller area, thus realizing more complex optical neural networks. Currently, the metasurface structure in metasurface-based optical neural networks is fixed, making dynamic control of light waves impossible. Therefore, their flexibility and adaptability are relatively poor when facing different task processing requirements.

[0004] Therefore, how to solve the above-mentioned technical problems should be a key focus for those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide an optical neural network training method, apparatus, electronic device, computer-readable storage medium, and optical neural network chip. The optical neural network has adjustable characteristics and can meet different task requirements.

[0006] To address the aforementioned technical problems, this invention provides an optical neural network training method, comprising:

[0007] Step S11: Under the current magnetic field state, the optical signal of the training image is transmitted through the metasurface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are controlled by the magnetic field.

[0008] Step S12: Transmit the optical signal passing through the metasurface structure to the activation layer, and make the activation layer simulate the activation function of the optical neural network according to the optical weight parameters;

[0009] Step S13: Determine the output error between the output result corresponding to the training image and the expected output result. The output result is output by the output layer when the optical signal transmitted to the output layer after passing through the activation function.

[0010] Step S14: When the output error does not meet the target output error requirement, adjust the network parameters according to the output error until the preset termination condition is met; the network parameters include the optical weight parameters.

[0011] Step S15: Adjust the magnetic field to change the polarization channel of the optical neural network, and proceed to step S11 to obtain the optical neural network under different polarization channels.

[0012] As one possible implementation, before transmitting the optical signal of the training image through the metasurface structure in the optical neural network to simulate the optical weight parameters of the training image under the current magnetic field state, the method further includes:

[0013] The size of the training image is adjusted to meet the input requirements of the metasurface structure.

[0014] The optical weight parameters of the metasurface structure and the initial strength of the magnetic field are initialized based on the adjusted size of the training image.

[0015] As one possible implementation, transmitting the optical signal passing through the metasurface structure to the activation layer, and having the activation layer simulate the activation function of the optical neural network according to the optical weight parameters includes:

[0016] The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the nonlinear activation function of the optical neural network according to the optical weight parameters.

[0017] As one possible implementation, transmitting the optical signal passing through the metasurface structure to a nonlinear activation layer, and having the nonlinear activation layer simulate the nonlinear activation function of the optical neural network according to the optical weight parameters includes:

[0018] The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the S-shaped growth curve function of the optical neural network according to the optical weight parameters.

[0019] As one possible implementation, transmitting the optical signal passing through the metasurface structure to a nonlinear activation layer, and having the nonlinear activation layer simulate the nonlinear activation function of the optical neural network according to the optical weight parameters includes:

[0020] The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the linear rectification function of the optical neural network according to the optical weight parameters.

[0021] As one possible implementation, adjusting the optical weighting parameters according to the output error until a preset termination condition is met includes:

[0022] Step S141: Determine a first gradient of the output error with respect to the optical weight parameters of each neuron in the optical neural network; wherein the first gradient represents the sensitivity of the output error to the optical weight parameters;

[0023] Step S142: Determine a second gradient of the output error with respect to the bias of each neuron in the optical neural network; wherein the second gradient represents the sensitivity of the output error to the bias;

[0024] Step S143: Based on the first gradient and the second gradient, the optical weight parameters and the bias of each neuron are adjusted using an optimization algorithm to obtain new optical weight parameters and new bias; wherein, the adjustment direction of the optical weight parameters is opposite to the direction of the first gradient; the network parameters include the optical weight parameters and the bias;

[0025] Step S144: When the output layer propagates backward to the input layer of the optical neural network layer by layer to obtain the trained optical neural network, the trained optical neural network is used as a new optical neural network and enters step S11 until a preset termination condition is reached. The preset termination condition includes the output error meeting the target output error requirement or the number of training times of the optical neural network reaching a preset number threshold.

[0026] The present invention also provides an optical neural network training device, comprising:

[0027] The first simulation module is used to execute step S11, which involves transmitting the optical signal of the training image through the metasurface structure in the optical neural network under the current magnetic field state to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are controlled by the magnetic field.

[0028] The second simulation module is used to execute step S12, which transmits the optical signal passing through the metasurface structure to the activation layer, and makes the activation layer simulate the activation function of the optical neural network according to the optical weight parameters.

[0029] The determination module is used to perform step S13, determining the output error between the output result corresponding to the training image and the expected output result, wherein the output result is output by the output layer when the optical signal transmitted to the output layer through the activation function;

[0030] The first adjustment module is used to execute step S14: when the output error does not meet the target output error requirement, adjust the optical weight parameters according to the output error until a preset termination condition is reached; the network parameters include the optical weight parameters.

[0031] The adjustment and iteration module is used to perform step S15, adjust the magnetic field to change the polarization channel of the optical neural network, and proceed to step S11 to obtain the optical neural network under different polarization channels.

[0032] The present invention also provides an electronic device, including

[0033] Memory, used to store computer programs;

[0034] A processor, configured to implement the steps of any of the above-described optical neural network training methods when executing the computer program.

[0035] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described optical neural network training methods.

[0036] The present invention also provides an optical neural network chip, trained using any of the optical neural network training methods described above, comprising an input layer, a hidden layer, a nonlinear activation layer, and an output layer, wherein the hidden layer comprises a multi-layer metasurface structure, and the characteristic parameters of the metasurface structure are controlled by a magnetic field.

[0037] The present invention provides an optical neural network training method, comprising: step S11, under the current magnetic field state, transmitting the optical signal of a training image through a metasurface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are controlled by the magnetic field; step S12, transmitting the optical signal through the metasurface structure to the activation layer, and causing the activation layer to simulate the activation function of the optical neural network according to the optical weight parameters; step S13, determining the output error between the output result corresponding to the training image and the expected output result, wherein the output result is output by the output layer when the optical signal through the activation function is transmitted to the output layer; step S14, when the output error does not meet the target output error requirement, adjusting the network parameters according to the output error until a preset termination condition is reached; the network parameters include the optical weight parameters; step S15, adjusting the magnetic field to change the polarization channel of the optical neural network, and proceeding to step S11 to obtain the optical neural network under different polarization channels.

[0038] As can be seen, the optical neural network in this invention is an optical neural network based on a metasurface structure. Under a given magnetic field condition, the training image is transmitted to the metasurface structure, and the optical weight parameters of the training image are simulated. Then, the optical signal transmitted through the metasurface structure is transmitted to the activation layer. The activation layer simulates the activation function of the optical neural network according to the optical weight parameters. The activation function processes the input optical signal, and the optical signal is then transmitted to the output layer, which outputs the result corresponding to the training image. The output error is determined based on the output result and the expected output result. The network parameters are then adjusted based on the output error until a preset termination condition is reached, completing the training under the current magnetic field. Changing the magnetic field causes changes in the feature parameters of the metasurface structure, thereby changing the polarization channel of the optical neural network, that is, changing the polarization state of the training image. Then, training is performed again to obtain optical neural networks with different polarization channels. Therefore, this invention can change the feature parameters of the metasurface structure by adjusting the magnetic field, thereby obtaining optical neural networks with different polarization channels and achieving dynamic adjustment of the optical signal. This dynamically adjustable optical neural network of this invention can meet the needs of various tasks and has high flexibility and adaptability.

[0039] Furthermore, the present invention also provides an apparatus, electronic device, computer-readable storage medium, and optical neural network chip having the above-mentioned advantages. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 The flowchart of an optical neural network training method provided in the embodiments of the present invention Figure 1 ;

[0042] Figure 2 The flowchart of an optical neural network training method provided in the embodiments of the present invention Figure 2 ;

[0043] Figure 3 The flowchart of an optical neural network training method provided in the embodiments of the present invention Figure 3 ;

[0044] Figure 4 This is a structural block diagram of an optical neural network training device provided in an embodiment of the present invention;

[0045] Figure 5A structural block diagram of an electronic device provided in an embodiment of the present invention;

[0046] Figure 6 A schematic diagram of an optical neural network model provided in an embodiment of the present invention;

[0047] Figure 7 This is a schematic diagram of a cascaded metasurface diffraction neural network for multiplexing information. Detailed Implementation

[0048] To enable those skilled in the art to better understand the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0050] As mentioned in the background section, the metasurface structure in current optical neural networks based on metasurface structures is fixed and cannot dynamically control light waves. Therefore, it has poor flexibility and adaptability when facing different task processing requirements.

[0051] In view of this, the present invention provides an optical neural network training method, please refer to... Figure 1 The method may include:

[0052] Step S11: Under the current magnetic field condition, the optical signal of the training image is transmitted through the metasurface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are controlled by the magnetic field.

[0053] The training image refers to the image used to train the optical neural network. This embodiment does not impose specific limitations and you can choose the image yourself.

[0054] The size of the training images in this step is matched to the size of the metasurface structure.

[0055] It should be noted that the current magnetic field state is not limited in this embodiment; it can be either a state with no magnetic field or a state with a magnetic field. Furthermore, when the magnetic field is present, the strength and direction of the magnetic field are not limited in this embodiment and can be set arbitrarily. Under the current magnetic field state, the optical neural network corresponds to one polarization channel.

[0056] The optical signal corresponds to the training image. The optical signal is input from the training image to the input layer of the optical neural network and then converted by the input layer.

[0057] Optical signals, also known as light waves, propagate in the forward direction. When incident on a metasurface structure, the metasurface structure can modulate the phase, amplitude, and polarization of the optical signal. The metasurface structure comprises multiple nanoantenna units, each possessing the characteristic of modulating light waves. Each nanoantenna unit can be considered a neuron within the metasurface structure.

[0058] The process of encoding the training image into the metasurface structure neurons in the input layer includes: a) Image preprocessing: Preprocessing the training image, including grayscale conversion, normalization, etc., to make the training image meet the input requirements of the optical neural network; b) Pixel mapping: Mapping each pixel of the training image to a metasurface structure neuron, with each pixel corresponding to one metasurface structure neuron; c) Amplitude or phase encoding: Each metasurface structure neuron performs phase encoding based on its corresponding pixel value. For example, the pixel value can be converted into the corresponding amplitude or phase value.

[0059] Before entering the metasurface structure, the training image is converted into an initial light field distribution. After entering the metasurface structure, the diffraction field distribution is calculated using the Fresnel diffraction integral formula. The diffracted light wave is then transmitted into the activation layer.

[0060] The Fresnel diffraction integral formula is:

[0061] (1)

[0062] In the formula, It is light waves in The diffraction field distribution at that location, It is light waves in The initial light field distribution at that location, It is the zeroth order Hankel function of the first kind. It is the light wave number. It is light waves from arrive distance, It is the imaginary unit.

[0063] The characteristic parameters of metasurface structures include, but are not limited to, refractive index, absorptivity, and birefringence. A magnetic field is applied to the metasurface structure, which is made of a magnetron sputtering material. When the magnetic field changes, the characteristic parameters of the metasurface structure change, thereby altering the modulation of optical signals.

[0064] The number of metasurface structures is not limited in this embodiment; it can be two, three, four, etc., with each layer of metasurface structure cascaded.

[0065] Step S12: The optical signal passing through the metasurface structure is transmitted to the activation layer, and the activation layer simulates the activation function of the optical neural network according to the optical weight parameters.

[0066] In the forward transmission direction of the optical signal, the nonlinear activation layer is located in the layer below the metasurface structure. The nonlinear activation layer can adjust its optical response according to the optical weight parameters of the optical signal output by the metasurface structure, simulating the activation function in the optical neural network, and further transforming and processing the optical signal.

[0067] It should be noted that the type of activation function is not limited in this embodiment and depends on the situation.

[0068] The activation layer can be an array of nanoantennas, which utilizes the local light field enhancement effect of the nanoantennas to generate an enhanced nonlinear response. The local light field enhancement effect of the nanoantennas can be controlled by adjusting the intensity of the input light, thus simulating the activation function in a neural network.

[0069] Step S13: Determine the output error between the output result corresponding to the training image and the expected output result. The output result is output by the output layer when the optical signal is transmitted to the output layer after passing through the activation function.

[0070] Step S14: When the output error does not meet the target output error requirement, adjust the network parameters according to the output error until the preset termination condition is met; the network parameters include optical weight parameters.

[0071] The preset termination condition can be that the output error meets the target output error requirement or the number of training iterations of the optical neural network reaches a preset threshold, etc. This embodiment does not impose specific limitations and can be set by the user.

[0072] It should be noted that this embodiment does not impose any restrictions on the target output error requirements, which can be set by the user.

[0073] By adjusting the network parameters to achieve the preset termination condition, an optical neural network trained under a certain polarization state is obtained.

[0074] Step S15: Adjust the magnetic field to change the polarization channel of the optical neural network, and proceed to step S11 to obtain the optical neural network under different polarization channels.

[0075] It should be noted that the adjustment of the magnetic field in this embodiment is not limited and depends on the situation. For example, it can be adjusted from having no magnetic field to having a magnetic field, or from having a magnetic field to having no magnetic field, or the strength and direction of the magnetic field can be changed when there is a magnetic field.

[0076] When the magnetic field changes, the characteristic parameters of the metasurface structure change, thereby altering the control of the phase, amplitude, and polarization of the optical signal, and achieving dynamic control of the light wave.

[0077] The characteristic parameters of the metasurface structure change, that is, they change between different polarization channels. Then, the optical neural network is trained under different polarization channels to obtain the optical neural network corresponding to the new polarization channel. Therefore, through training in this embodiment, optical neural networks trained under different polarization channels can be obtained. When facing different processing tasks, an optical neural network under a specific polarization channel can be selected for task processing according to different task requirements, making the optical neural network highly flexible and adaptable.

[0078] The trained optical neural network can be used for tasks such as optical image recognition and classification. Correspondingly, the output of the optical neural network includes recognition results, classification results, etc.

[0079] In this embodiment, the optical neural network is an optical neural network based on a metasurface structure. Under a given magnetic field condition, a training image is transmitted to the metasurface structure, and the optical weight parameters of the training image are simulated. Then, the optical signal transmitted through the metasurface structure is transmitted to the activation layer. The activation layer simulates the activation function of the optical neural network according to the optical weight parameters. The activation function then outputs an output result corresponding to the training image. The output error is determined based on the output result of the activation function and the expected output result. The network parameters are then adjusted based on the output error until a preset termination condition is reached, completing the training under the current magnetic field. Changing the magnetic field causes changes in the feature parameters of the metasurface structure, thereby changing the polarization channel of the optical neural network, that is, changing the polarization state of the training image. Then, training is performed again to obtain optical neural networks with different polarization channels. Therefore, this invention can change the feature parameters of the metasurface structure by adjusting the magnetic field, thereby obtaining optical neural networks with different polarization channels and realizing dynamic adjustment of optical signals. This dynamically adjustable optical neural network of the present invention can meet the needs of various tasks and has high flexibility and adaptability.

[0080] Please refer to Figure 2 Based on the above embodiments, in one embodiment of the present invention, the optical neural network training method may include:

[0081] Step S21: Adjust the size of the training image so that the training image meets the input requirements of the metasurface structure.

[0082] When the size of the training image does not match the metasurface structure, the size of the input training image needs to be preprocessed to match the size of the training image with the metasurface structure.

[0083] When the size of the input training image does not match the size of the metasurface structure, the following methods can be used to address this issue.

[0084] (1) Scaling: When the size of the training image is M×N, and the size of the training image is larger than the size of the metasurface structure, the training image is scaled to match the size of the training image with that of the metasurface structure. For example, the training image can be scaled proportionally or divided into multiple parts for processing.

[0085] (2) Sampling: The training image is sampled to convert it into a number of pixels that matches the size of the metasurface structure. For example, the training image can be downsampled or upsampled.

[0086] (3) Padding: When the size of the training image is M×N, and the size of the training image is smaller than the size of the metasurface structure, the training image is padded to match the size of the metasurface structure. For example, blank pixels can be added around the image or a padding algorithm can be used.

[0087] Step S22: Based on the adjusted size of the training image, initialize the optical weight parameters of the metasurface structure and the initial strength of the magnetic field.

[0088] The initial optical weight parameters and the initial magnetic field strength are not limited in this embodiment and can be set by the user. When the magnetic field is at the initial strength, the polarization channel corresponding to the optical neural network can be called the first polarization channel. Each time the magnetic field is adjusted, the polarization channel changes to a new polarization channel, which are sequentially called the second polarization channel, the third polarization channel, and so on.

[0089] Step S23: Under the current magnetic field condition, the optical signal of the training image is transmitted through the metasurface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are controlled by the magnetic field.

[0090] Understandably, during the initial training, the current magnetic field is the initial magnetic field strength, corresponding to the first polarization channel of the optical neural network. After the magnetic field is adjusted, the polarization channel changes, and the optical neural network is trained again under the corresponding new polarization channel.

[0091] Step S24: Transmit the optical signal through the metasurface structure to the activation layer, and make the activation layer simulate the activation function of the optical neural network according to the optical weight parameters.

[0092] In the forward propagation direction of the optical signal, the nonlinear activation layer is located in the layer below the metasurface structure. The nonlinear activation layer can adjust its optical response according to the optical weight parameters of the optical signal output by the metasurface structure, simulating the activation function in the optical neural network, and further transforming and processing the optical signal.

[0093] Step S25: Determine the output error between the output result corresponding to the training image and the expected output result. The output result is output by the output layer when the optical signal is transmitted to the output layer after passing through the activation function.

[0094] Step S26: Adjust the network parameters according to the output error until the preset termination condition is met.

[0095] Step S27: Adjust the magnetic field to change the polarization channel of the optical neural network, and proceed to step S23 to obtain the optical neural network under different polarization channels.

[0096] It should be noted that steps S23 to S27 can be referred to the content of the above embodiments, and will not be described in detail here.

[0097] Based on any of the above embodiments, in one embodiment of the present invention, after obtaining the optical neural network under different polarization channels, the following may be further included:

[0098] The training samples are used to evaluate the performance of the trained optical neural network, and the evaluation results such as classification accuracy, training time and energy consumption are determined. The optical neural network is then optimized and adjusted based on the evaluation results.

[0099] Please refer to Figure 3 Based on any of the above embodiments, in one embodiment of the present invention, the optical neural network training method may include:

[0100] Step S31: Under the current magnetic field condition, the optical signal of the training image is transmitted through the metasurface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are controlled by the magnetic field.

[0101] Step S32: Transmit the optical signal through the metasurface structure to the nonlinear activation layer, and make the nonlinear activation layer simulate the nonlinear activation function of the optical neural network according to the optical weight parameters.

[0102] The nonlinear activation layer can be an array composed of nanoantenna structures with nonlinear optical effects. The material of the nonlinear activation layer can be a nonlinear optical material, including but not limited to lithium niobate, lithium tantalate, liquid crystal, etc.

[0103] The enhanced nonlinear response is generated by utilizing the local optical field enhancement effect of nanoantennas, and the local optical field enhancement effect of nanoantennas is controlled by adjusting the intensity of input light, thus simulating the nonlinear activation function in optical neural networks.

[0104] In this embodiment, the activation function of the simulated optical neural network is a nonlinear activation function. The nonlinear activation function can introduce nonlinear factors into the optical neural network model, allowing the optical neural network to arbitrarily approximate the nonlinear function, so that the optical neural network can be applied to many nonlinear models.

[0105] It should be noted that the type of nonlinear activation function is not limited in this embodiment, and users can choose it themselves.

[0106] As one possible implementation, the optical signal passing through the metasurface structure is transmitted to a nonlinear activation layer, and the nonlinear activation layer simulates the nonlinear activation function of an optical neural network according to optical weight parameters, including:

[0107] The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the S-shaped growth curve function of the optical neural network according to the optical weight parameters.

[0108] The S-shaped growth curve function can be the Sigmoid function. The Sigmoid function's output is in the range [0,1], making it particularly suitable for models that output probabilities. Since any probability takes values ​​between 0 and 1, the Sigmoid activation function is the best choice.

[0109] As another possible implementation, the optical signal passing through the metasurface structure is transmitted to a nonlinear activation layer, and the nonlinear activation layer simulates the nonlinear activation function of an optical neural network according to optical weight parameters, including:

[0110] The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the linear rectification function of the optical neural network according to the optical weight parameters.

[0111] The linear rectified function can be the ReLU (Rectified Linear Unit) function. The ReLU function has advantages such as simple calculation, fast convergence speed, reduced overfitting, and mitigation of gradient vanishing.

[0112] When the intensity of the input optical signal is below a preset threshold, the nonlinear response is small, similar to the linear part of the ReLU function; when the intensity of the input optical signal exceeds the preset threshold, the nonlinear response increases significantly, simulating the saturation part of the ReLU function or the saturation characteristics of the Sigmoid function.

[0113] Step S33: Determine the output error between the output result corresponding to the training image and the expected output result. The output result is output by the output layer when the optical signal is transmitted to the output layer through the nonlinear activation function.

[0114] Step S34: When the output error does not meet the target output error requirement, adjust the network parameters according to the output error until the preset termination condition is met.

[0115] Step S35: Adjust the magnetic field to change the polarization channel of the optical neural network, and proceed to step S31 to obtain the optical neural network under different polarization channels.

[0116] It should be noted that steps S31, S33 to S25 can be referred to the content of the above embodiments, and will not be described in detail here.

[0117] Based on any of the above embodiments, in one embodiment of the present invention, adjusting the network parameters according to the output error until a preset termination condition is reached includes:

[0118] Step S141: Determine the first gradient of the output error with respect to the optical weight parameters of each neuron in the optical neural network; where the first gradient represents the sensitivity of the output error to the optical weight parameters.

[0119] Each layer in an optical neural network contains neurons.

[0120] The first gradient is used to guide the direction of adjustment of the optical weight parameters.

[0121] Step S142: Determine the second gradient of the output error with respect to the bias of each neuron in the optical neural network; where the second gradient represents the sensitivity of the output error to the bias.

[0122] Step S143: Based on the first gradient and the second gradient, the optical weight parameters and biases of each neuron are adjusted using an optimization algorithm to obtain new optical weight parameters and new biases; wherein, the adjustment direction of the optical weight parameters is opposite to the direction of the first gradient; the network parameters include optical weight parameters and biases.

[0123] This embodiment does not limit the type of optimization algorithm; any algorithm can be selected. As one possible implementation method, the optimization algorithm includes, but is not limited to, any one of the following: gradient descent, momentum method, adaptive learning rate method, and regularization method.

[0124] Adjusting the optical weight parameters in the opposite direction to the first gradient increases the weights to reduce output error.

[0125] Step S144: When the output layer propagates backward to the input layer of the optical neural network, the trained optical neural network is obtained. The trained optical neural network is used as a new optical neural network and the process proceeds to step S11 until the preset termination condition is reached. The preset termination condition includes the output error meeting the target output error requirement or the number of training iterations of the optical neural network reaching a preset threshold.

[0126] In this embodiment, the preset number of times threshold is not limited and can be set by the user.

[0127] Backpropagation proceeds layer by layer, in the following order: output layer, activation layer, hidden layer, and input layer. Once all weight parameters and biases have been adjusted (i.e., updated), the optical neural network has completed one training iteration. After multiple repetitions, the performance of the optical neural network can be trained to its optimal level under a specific magnetic field condition, thereby improving the accuracy of task processing.

[0128] Based on any of the above embodiments, in one embodiment of the present invention, determining the output error between the output result corresponding to the training image and the expected output result includes:

[0129] The difference between the output result and the expected output result is calculated using the loss function, and this difference is taken as the output error.

[0130] As one possible implementation, the cross-entropy loss function is used to calculate the difference between the output result and the expected output result to obtain the output error.

[0131] In classification problems, especially multi-class classification problems, the cross-entropy loss function can provide good gradient properties, which helps the model converge quickly.

[0132] As another possible implementation method, the difference between the output result and the expected output result is calculated using the mean squared error loss function to obtain the output error.

[0133] The mean squared error loss function can intuitively measure the square of the difference between the output result and the expected output result, which is easy to understand and interpret; it is simple and direct to calculate and fast; the gradient of the mean squared error loss function is continuous and smooth, which helps the gradient descent algorithm to converge stably.

[0134] The optical neural network training device provided in the embodiments of the present invention will be described below. The optical neural network training device described below can be referred to in correspondence with the optical neural network training method described above.

[0135] Figure 4 This is a structural block diagram of the optical neural network training device provided in an embodiment of the present invention, with reference to... Figure 4 Optical neural network training devices may include:

[0136] The first simulation module 100 is used to execute step S11, which involves transmitting the optical signal of the training image through the metasurface structure in the optical neural network under the current magnetic field state to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are controlled by the magnetic field.

[0137] The second simulation module 200 is used to execute step S12, which transmits the optical signal passing through the metasurface structure to the activation layer, and makes the activation layer simulate the activation function of the optical neural network according to the optical weight parameters.

[0138] The determination module 300 is used to perform step S13, determining the output error between the output result corresponding to the training image and the expected output result. The output result is output by the output layer when the optical signal transmitted to the output layer after passing through the activation function.

[0139] The first adjustment module 400 is used to execute step S14, adjusting the network parameters according to the output error until a preset termination condition is met; the network parameters include optical weight parameters.

[0140] The adjustment and iteration module 500 is used to execute step S15, adjust the magnetic field to change the polarization channel of the optical neural network, and proceed to step S11 to obtain the optical neural network under different polarization channels.

[0141] The optical neural network training device of this embodiment is used to implement the aforementioned optical neural network training method. Therefore, the specific implementation of the optical neural network training device can be found in the embodiment section of the optical neural network training method above. For example, the first simulation module 100, the second simulation module 200, the determination module 300, the first adjustment module 400, and the adjustment and iteration module 500 are respectively used to implement steps S11, S12, S13, S14 and S15 in the above-mentioned optical neural network training method. Therefore, its specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.

[0142] In this embodiment of the optical neural network training device, a training image is transmitted to a metasurface structure under a current magnetic field state. The optical weight parameters of the training image are simulated. Then, the optical signal transmitted through the metasurface structure is transmitted to the activation layer. The activation layer simulates the activation function of the optical neural network based on the optical weight parameters. The activation function then outputs an output result corresponding to the training image. The output error is determined based on the output result of the activation function and the expected output result. The optical weight parameters are then adjusted based on the output error until the error meets the target output error requirement, completing the training under the current magnetic field. Changing the magnetic field alters the characteristic parameters of the metasurface structure, thereby changing the polarization channel of the optical neural network, i.e., changing the polarization state of the training image. Training is then performed again to obtain optical neural networks with different polarization channels. Therefore, this invention can change the characteristic parameters of the metasurface structure by adjusting the magnetic field, thereby obtaining optical neural networks with different polarization channels and achieving dynamic adjustment of the optical signal. This dynamically adjustable optical neural network of the present invention can meet the needs of various tasks and has high flexibility and adaptability.

[0143] As one possible implementation, the optical neural network training device may further include:

[0144] The second adjustment module is used to adjust the size of the training image so that the training image meets the input requirements of the metasurface structure.

[0145] The initialization module is used to initialize the optical weight parameters of the metasurface structure and the initial strength of the magnetic field based on the adjusted size of the training image.

[0146] As one possible implementation, the second simulation module 200 is specifically used to transmit the optical signal passing through the metasurface structure to the nonlinear activation layer, and to make the nonlinear activation layer simulate the nonlinear activation function of the optical neural network according to the optical weight parameters.

[0147] As one possible implementation, the second simulation module 200 is specifically used to transmit the optical signal passing through the metasurface structure to the nonlinear activation layer, and to make the nonlinear activation layer simulate the S-shaped growth curve function of the optical neural network according to the optical weight parameters.

[0148] As one possible implementation, the second simulation module 200 is specifically used to transmit the optical signal passing through the metasurface structure to the nonlinear activation layer, and to make the nonlinear activation layer simulate the linear rectification function of the optical neural network according to the optical weight parameters.

[0149] As one possible implementation, the adjustment and iteration module 500 includes:

[0150] The first determining unit is configured to perform step S141: determining a first gradient of the output error with respect to the optical weight parameters of each neuron in the optical neural network; wherein the first gradient represents the sensitivity of the output error to the optical weight parameters;

[0151] The second determining unit is used in step S142 to determine a second gradient of the output error with respect to the bias of each neuron in the optical neural network; wherein the second gradient represents the sensitivity of the output error to the bias.

[0152] An adjustment unit is used in step S143 to: adjust the optical weight parameters and the bias of each neuron using an optimization algorithm based on the first gradient and the second gradient, to obtain new optical weight parameters and new bias; wherein the adjustment direction of the optical weight parameters is opposite to the direction of the first gradient; the network parameters include the optical weight parameters and the bias;

[0153] The repeating unit is used to execute step S144: when the output layer propagates backward to the input layer of the optical neural network to obtain the trained optical neural network, the trained optical neural network is used as a new optical neural network and enters step S11 until a preset termination condition is reached, wherein the preset termination condition includes the output error meeting the target output error requirement or the number of training times of the optical neural network reaching a preset number threshold.

[0154] The electronic device provided in the embodiments of the present invention will be described below. The electronic device described below can be referred to in correspondence with the optical neural network training method described above.

[0155] Figure 5 The structural block diagram of the electronic device provided in the embodiments of the present invention includes...

[0156] Memory 11 is used to store computer programs;

[0157] The processor 12 is used to implement the steps of the optical neural network training method of any of the above embodiments when executing a computer program.

[0158] The following describes the computer-readable storage medium provided in the embodiments of the present invention. The computer-readable storage medium described below can be referred to in correspondence with the optical neural network training method described above.

[0159] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the optical neural network training method of any of the above embodiments.

[0160] The present invention also provides an optical neural network chip, which is trained using the optical neural network training method of any of the above embodiments.

[0161] Please refer to Figure 6 The optical neural network chip includes an input layer 1, a hidden layer 2, a nonlinear activation layer 3, and an output layer 4. The hidden layer 2 includes a multi-layer metasurface structure, and the characteristic parameters of the metasurface structure are controlled by a magnetic field.

[0162] Input layer 1 is used to convert the received image into an optical signal and map the image information into the intensity or phase of light.

[0163] The number of metasurface structures in hidden layer 2 is not limited in this embodiment and can be set by the user.

[0164] The schematic diagram of the cascaded metasurface diffraction neural network for multiplexing multiplexed information is as follows: Figure 7 As shown, a diffractive optical neural network with multiplexed information can be constructed by cascading multiple metasurface structures. This network can simultaneously process and identify input images from different polarization channels and transmit the input images to a specific detection area integrating multiple detectors, thereby realizing the recognition and classification of different images.

[0165] Metasurface structures consist of subwavelength scattering units with specific shapes. These scattering units produce light diffraction effects, which can modulate the phase, amplitude, and polarization of incident light waves, thereby enabling the encoding and processing of image information.

[0166] The material of the metasurface structure is a magnetron material. Under the action of a magnetic field, the characteristic parameters such as the refractive index, absorptivity and birefringence of the magnetron material change, so the phase, amplitude and polarization modulation state of the scattering unit on the incident light wave can be dynamically adjusted.

[0167] When an image from multiple polarization channels is input into a metasurface structure composed of an array of magnetic scattering units, each nanoantenna unit (scattering unit) will generate polarization modulation characteristics under the action of an external magnetic field, enabling the metasurface structure to precisely control light waves in different polarization states.

[0168] The nonlinear activation layer 3 can be an array composed of nanoantenna structures with nonlinear optical effects. The material of the nonlinear activation layer can be a nonlinear optical material, including but not limited to lithium niobate, lithium tantalate, liquid crystal, etc. Enhanced nonlinear response is generated by utilizing the local optical field enhancement effect of the nanoantennas, and the local optical field enhancement effect of the nanoantennas is controlled by adjusting the intensity of the input light, thus simulating the nonlinear activation function in an optical neural network.

[0169] Output layer 4 is used to output the results.

[0170] The optical neural network in this embodiment is a reconfigurable all-optical neural network based on metasurfaces. Compared with traditional electronic neural networks, all-optical neural networks can achieve faster information processing by utilizing the high-speed propagation characteristics of light. This high-speed processing capability is of great significance for processing large-scale, complex datasets and application scenarios with high real-time requirements.

[0171] In this embodiment, the optical neural network can achieve complex optical transformations and information processing by cascading multiple magnetically controlled metasurface structures, giving the all-optical neural network the ability to perform multiple tasks and making it suitable for use in multifunctional and efficient information processing systems.

[0172] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0173] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0174] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0175] The optical neural network training method, apparatus, device, medium, and optical neural network chip provided by this invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the solution and core ideas of this invention. It should be noted that those skilled in the art can make several improvements and modifications to this invention without departing from the principles of this invention, and these improvements and modifications also fall within the protection scope of this invention.

Claims

1. A method for training an optical neural network, characterized in that, include: Step S11: Under the current magnetic field state, the optical signal of the training image is transmitted through the metasurface structure in the optical neural network to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are controlled by the magnetic field. Step S12: Transmit the optical signal passing through the metasurface structure to the activation layer, and make the activation layer simulate the activation function of the optical neural network according to the optical weight parameters; Step S13: Determine the output error between the output result corresponding to the training image and the expected output result. The output result is output by the output layer when the optical signal transmitted to the output layer after passing through the activation function. Step S14: When the output error does not meet the target output error requirement, adjust the network parameters according to the output error until the preset termination condition is met; the network parameters include the optical weight parameters. Step S15: Adjust the magnetic field to change the polarization channel of the optical neural network, and proceed to step S11 to obtain the optical neural network under different polarization channels.

2. The optical neural network training method as described in claim 1, characterized in that, Before transmitting the optical signal of the training image through the metasurface structure in the optical neural network under the current magnetic field state to simulate the optical weight parameters of the training image, the process further includes: The size of the training image is adjusted to meet the input requirements of the metasurface structure. The optical weight parameters of the metasurface structure and the initial strength of the magnetic field are initialized based on the adjusted size of the training image.

3. The optical neural network training method as described in claim 1, characterized in that, Transmitting the optical signal passing through the metasurface structure to the activation layer, and having the activation layer simulate the activation function of the optical neural network according to the optical weight parameters, includes: The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the nonlinear activation function of the optical neural network according to the optical weight parameters.

4. The optical neural network training method as described in claim 3, characterized in that, Transmitting the optical signal passing through the metasurface structure to the nonlinear activation layer, and having the nonlinear activation layer simulate the nonlinear activation function of the optical neural network according to the optical weight parameters, includes: The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the S-shaped growth curve function of the optical neural network according to the optical weight parameters.

5. The optical neural network training method as described in claim 3, characterized in that, Transmitting the optical signal passing through the metasurface structure to the nonlinear activation layer, and having the nonlinear activation layer simulate the nonlinear activation function of the optical neural network according to the optical weight parameters, includes: The optical signal passing through the metasurface structure is transmitted to the nonlinear activation layer, and the nonlinear activation layer simulates the linear rectification function of the optical neural network according to the optical weight parameters.

6. The optical neural network training method according to any one of claims 1 to 5, characterized in that, Adjusting network parameters based on the output error until a preset termination condition is met includes: Step S141: Determine a first gradient of the output error with respect to the optical weight parameters of each neuron in the optical neural network; wherein the first gradient represents the sensitivity of the output error to the optical weight parameters; Step S142: Determine a second gradient of the output error with respect to the bias of each neuron in the optical neural network; wherein the second gradient represents the sensitivity of the output error to the bias; Step S143: Based on the first gradient and the second gradient, the optical weight parameters and the bias of each neuron are adjusted using an optimization algorithm to obtain new optical weight parameters and new bias; wherein, the adjustment direction of the optical weight parameters is opposite to the direction of the first gradient; the network parameters include the optical weight parameters and the bias; Step S144: When the output layer propagates backward to the input layer of the optical neural network layer by layer to obtain the trained optical neural network, the trained optical neural network is used as a new optical neural network and enters step S11 until a preset termination condition is reached. The preset termination condition includes the output error meeting the target output error requirement or the number of training times of the optical neural network reaching a preset number threshold.

7. An optical neural network training device, characterized in that, include: The first simulation module is used to execute step S11, which involves transmitting the optical signal of the training image through the metasurface structure in the optical neural network under the current magnetic field state to simulate the optical weight parameters of the training image; the characteristic parameters of the metasurface structure are controlled by the magnetic field. The second simulation module is used to execute step S12, which transmits the optical signal passing through the metasurface structure to the activation layer, and makes the activation layer simulate the activation function of the optical neural network according to the optical weight parameters. The determination module is used to perform step S13, determining the output error between the output result corresponding to the training image and the expected output result, wherein the output result is output by the output layer when the optical signal transmitted to the output layer through the activation function; The first adjustment module is used to execute step S14: when the output error does not meet the target output error requirement, adjust the network parameters according to the output error until a preset termination condition is reached; the network parameters include the optical weight parameters. The adjustment and iteration module is used to perform step S15, adjust the magnetic field to change the polarization channel of the optical neural network, and proceed to step S11 to obtain the optical neural network under different polarization channels.

8. An electronic device, characterized in that, include Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the optical neural network training method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the optical neural network training method as described in any one of claims 1 to 6.

10. An optical neural network chip, characterized in that, The optical neural network is trained using the training method described in any one of claims 1 to 6, and includes an input layer, a hidden layer, a nonlinear activation layer, and an output layer, wherein the hidden layer includes a multi-layer metasurface structure, and the characteristic parameters of the metasurface structure are controlled by a magnetic field.

Images