Optical neural network and architecture determination method, preparation method and preparation system thereof
By iteratively adjusting the optical component parameters of the optical modulation layer and the diffraction layer, and optimizing the optical neural network architecture, the accuracy problem of optical neural networks when modulating and identifying complex light fields is solved, and high-precision light field recognition and modulation are achieved, which is suitable for tasks such as image recognition and computational imaging.
Patent Information
- Application Number
- CN202510659167.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-21
- Publication Date
- 2025-07-25
AI Technical Summary
Existing optical neural networks are difficult to accurately shape the light field distribution when modulating complex light fields, and the signal processing and feature extraction of light field recognition links are difficult, resulting in low recognition accuracy.
By iteratively adjusting the optical element parameters of the optical modulation layer and the diffraction layer, and according to the output errors of the exit light field and the target light field, until the error meets the preset conditions, the optical neural network architecture is optimized and the recognition accuracy is improved.
It improves the recognition accuracy and robustness of optical neural networks, reduces the difficulty of manufacturing error alignment, enhances environmental adaptability, and is suitable for tasks such as image recognition and computational imaging.
Smart Images
Figure CN120373383A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of optics, and more particularly, to an optical neural network, a method for determining its architecture, a manufacturing method, and a manufacturing system. Background Art
[0002] With the rapid development of artificial intelligence technology, neural networks, as the core algorithm architecture, have demonstrated excellent performance and broad application prospects in many fields such as image recognition, speech processing, and natural language understanding. Traditional electronic neural networks are based on the von Neumann architecture and achieve data operations through frequent interactions between the processor and the memory by electrical signals. Although significant progress has been made in computing power and algorithm optimization, when dealing with large-scale and high-dimensional data, they still face bottleneck problems such as high energy consumption and limited computing speed.
[0003] To break through these limitations, optical neural networks have emerged as a new type of computing architecture. It makes full use of the high-speed parallel transmission, low energy consumption, and unique physical properties such as interference and diffraction of photons, combines optical elements with neural network models, and aims to achieve more efficient and lower-power data processing. In theory, optical neural networks have the ability to achieve ultra-high-speed and large-scale parallel computing, and are expected to bring new changes to the field of artificial intelligence.
[0004] However, current optical neural networks still face many challenges in practical applications. Among them, the low modulation and recognition accuracy of complex input optical fields is one of the key problems restricting their development. Existing optical modulation technologies often have difficulty precisely shaping the required optical field distribution when modulating complex optical fields. At the same time, in the optical field recognition link, signal processing and feature extraction are relatively difficult, reducing the recognition accuracy. Summary of the Invention
[0005] In view of this, the purpose of the embodiments of the present application is to provide an optical neural network, a method for determining its architecture, a manufacturing method, and a manufacturing system, which can improve the recognition accuracy of the optical neural network.
[0006] In a first aspect, the embodiments of the present application provide a method for determining an optical neural network architecture, including: determining the output optical field after the incident optical field passes through the optical modulation layer and / or the diffraction layer in the optical neural network; calculating the output error between the output optical field and the target optical field; adjusting the optical element parameters of the optical modulation layer according to the output error; determining the output optical field after the incident optical field passes through the optical modulation layer with adjusted optical element parameters and / or the diffraction layer, and determining the output error between the output optical field and the target optical field until the output error meets a preset condition; determining the optical element parameters corresponding to the output error that meets the preset condition as the target optical element parameters of the optical modulation layer.
[0007] In the above implementation process, by iteratively adjusting the optical element parameters of the optical modulation layer according to the output error between the outgoing light field after passing through the optical modulation layer and / or the diffraction layer and the target light field until the output error meets the preset error condition, the output error of the optical neural network can be reduced, thereby improving the recognition accuracy of the optical neural network.
[0008] In one embodiment, the optical neural network includes multiple structural layers, and each structural layer includes a corresponding optical modulation layer and / or diffraction layer; the determining of the outgoing light field after the incident light field passes through the optical modulation layer and / or the diffraction layer in the optical neural network includes: calculating the intermediate light field after the incident light field passes through the optical modulation layer and / or the diffraction layer in each structural layer; determining the intermediate light field output by the last structural layer as the outgoing light field.
[0009] In the above implementation process, when calculating the outgoing light field, specifically calculating the light field output by each structural layer respectively can improve the accuracy of calculating the outgoing light field.
[0010] In one embodiment, the calculating of the intermediate light field after the incident light field passes through the optical modulation layer and / or the diffraction layer in each structural layer includes: for the first structural layer, determining the light field modulated by the optical modulation layer in the first structural layer as the intermediate light field output by the first structural layer; for other structural layers, determining the intermediate light field output by the current structural layer through the first light field transmitted through the diffraction layer in the previous structural layer and the second light field modulated by the optical modulation layer in the current structural layer; wherein, in the optical neural network, the intermediate light field output by the previous structural layer serves as the input light field for the next structural layer.
[0011] In the above implementation process, when calculating the outgoing light field, respectively calculating the intermediate light field after the incident light field passes through the optical modulation layer and the diffraction layer in each structural layer, and then using the intermediate light field output by the previous structural layer as the input light field for the next structural layer, and specifically calculating the light field output by each structural layer respectively can achieve precise recognition and modulation of a complex incident light field, thereby improving the accuracy of calculating the outgoing light field.
[0012] In one embodiment, the adjusting of the optical element parameters of the optical modulation layer according to the output error includes: calculating the gradient of the loss function with respect to the optical element parameters in each structural layer through the chain rule; wherein, the loss function is used to calculate the output error between the outgoing light field and the target light field; determining the contribution value of each structural layer to the outgoing light field through the backpropagation algorithm; adjusting the optical element parameters of the optical modulation layer in each structural layer according to the optimization algorithm and the contribution value of each structural layer to the outgoing light field.
[0013] In the above implementation process, by adjusting the parameters of the optical element according to the output error, the accuracy of the optical element parameters can be improved. In addition, the optical key parameter adjustment method in the above embodiment is a training method designed specifically for a monolithic multi-layer large-area optical neural network. Considering the compensation for the manufacturing output error, the robustness of the system to manufacturing defects can be enhanced through structural redundancy and / or algorithm optimization, and the accuracy of the optical neural network architecture can be improved.
[0014] In one embodiment, calculating the intermediate light field after the incident light field passes through the optical modulation layer and / or the diffraction layer in each structural layer includes: incorporating the actual manufacturing error distribution into the diffraction theory and performing parameter optimization; calculating the intermediate light field after the incident light field passes through the optical modulation layer and / or the diffraction layer in each structural layer according to the diffraction theory.
[0015] In the above implementation process, during the parameter optimization process, introducing the actual manufacturing error can obtain an optical neural network architecture considering the actual manufacturing error, avoiding the problem of low accuracy of the manufactured optical neural network caused by the introduction of errors during the actual manufacturing process of the optical neural network, improving the tolerance of the model to errors, and further improving the accuracy of the optical neural network.
[0016] In a second aspect, an embodiment of the present application further provides an optical neural network manufacturing method, including: determining the processing parameters for manufacturing the optical neural network according to the target optical element parameters determined by the method in the first aspect or any one of the embodiments in the first aspect; manufacturing the optical neural network inside the medium through the processing parameters.
[0017] In the above implementation process, by determining the processing parameters according to the target optical element parameters, the accuracy of the processing parameters can be improved. In addition, manufacturing the optical neural network in a single medium does not require subsequent high-precision alignment, has a high integration level, can reduce the manufacturing difficulty of the optical neural network, and improve the manufacturing efficiency and accuracy.
[0018] In one embodiment, the target optical element parameters include: operating wavelength, interlayer distance, phase distribution, and / or amplitude distribution; the processing parameters include: processing energy, processing polarization, and processing position; determining the processing parameters for manufacturing the optical neural network according to the target optical element parameters determined by the method in the first aspect or any one of the embodiments in the first aspect includes: determining the processing energy and processing polarization according to the phase distribution and / or the amplitude distribution, and the operating wavelength; determining the processing position according to the interlayer distance.
[0019] In the above implementation process, by determining the processing energy and processing polarization according to the phase distribution and / or amplitude distribution, and the working wavelength, and determining the processing position according to the interlayer distance, the laser can process the optical neural network according to the corresponding processing parameters, so that the optical element parameters of each optical element of the processed optical neural network are all target optical element parameters, improving the processing accuracy.
[0020] In a third aspect, an embodiment of the present application further provides an optical neural network preparation system, including: a laser, an energy control optical path, a polarization control optical path, an objective lens, and a moving platform; the moving platform is used to place the device to be processed; the laser is used to emit a laser beam; the moving platform is used to adjust the processing position of the device to be processed; the energy control optical path is used to adjust the processing energy of the laser beam; the polarization control optical path is used to adjust the processing polarization of the processing energy in real time; wherein, the processing position, the processing energy, and the processing polarization are determined according to the target optical element parameters determined by the method in the first aspect or any embodiment of the first aspect; the objective lens is used to focus the laser beam into the inside of a single piece of medium; wherein, the laser beam performs input layer, output layer, and structure layer processing inside the single piece of medium to form an optical neural network.
[0021] In the above implementation process, by adopting the laser multi-layer processing technology, and the optical modulation layer is processed inside a single piece of medium, that is, multiple optical modulation layers are processed inside the same piece of medium, high-precision alignment is achieved during processing between layers, and it is more integrated, which can improve the environmental adaptability of the optical neural network. In addition, since the system does not rely on circuit signal processing and its monolithic structure, it can still work normally under strong electromagnetic interference or extreme temperature environments, increasing the application scenarios of the optical neural network preparation system.
[0022] In a fourth aspect, an embodiment of the present application further provides an optical neural network, including: an input layer, an output layer, and a structure layer; the structure layer is arranged between the input layer and the output layer; wherein, the structure layer includes one or more layers; the layers of the input layer, the structure layer, and the output layer are connected through diffraction theory; the input layer is configured to receive an incident light field; the structure layer is configured to modulate the incident light field; the output layer is configured to output the light field modulated by the structure layer; wherein, the optical element parameters of the optical modulation layer in the structure layer are determined by the method in the first aspect or any embodiment of the first aspect.
[0023] In the above implementation process, through the multi-layer structural layer, the incident light field can be modulated layer by layer to achieve precise recognition and modulation of complex incident light fields. In addition, the recognition accuracy depends on the number of layers of the optical neural network and the design of the optical modulation layer of each layer, enabling the optical neural network to capture subtle feature changes in the input, being applicable to various tasks such as image recognition and computational imaging, and increasing the application scenarios of the optical neural network.
[0024] In one embodiment, the input layer, the output layer, and the structural layer are disposed inside a single-piece medium.
[0025] In the above implementation process, by processing multi-layer structural layers inside the same piece of medium, high-precision alignment is achieved during processing between layers, and it is more integrated, improving the environmental adaptability of the optical neural network.
[0026] To make the above objects, features, and advantages of the present application more obvious and understandable, specific embodiments are hereinafter given, and in conjunction with the accompanying drawings, the detailed description is as follows. Description of the Drawings
[0027] To more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative efforts.
[0028] Figure 1 Flowchart of the method for determining the optical neural network architecture provided by the embodiment of the present application;
[0029] Figure 2 Iterative schematic diagram in the method for determining the optical neural network architecture provided by the embodiment of the present application;
[0030] Figure 3 Flowchart of the method for fabricating the optical neural network provided by the embodiment of the present application;
[0031] Figure 4 Structural schematic diagram of the optical neural network fabrication system provided by the embodiment of the present application;
[0032] Figure 5 Structural schematic diagram of the optical neural network provided by the embodiment of the present application.
[0033] Description of the Drawings: 100 - Laser, 200 - Energy control optical path, 300 - Polarization control optical path, 400 - Objective lens, 500 - Moving platform. Detailed Embodiments
[0034] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.
[0035] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, the terms "first", "second", etc. are only used for distinguishing descriptions and cannot be construed as indicating or implying relative importance.
[0036] As one of the core technologies in the field of artificial intelligence, the scale of neural networks is constantly expanding, and the amount of data to be processed is also increasing. Currently, neural networks mainly run on traditional electronic computers. However, the imbalance between the growth rate of the computing power required for neural network model training and the update rate of electronic devices has become an important factor restricting its development. Optical neural networks load computing data through optical signals and perform calculations using optical modulation devices in combination with the diffraction theory of light. Compared with neural networks based on traditional electronic technologies, optical neural networks have many advantages such as higher neuron density, faster computing speed, stronger parallel processing ability, and lower energy consumption.
[0037] Optical neural networks are mainly concentrated in the terahertz band and the visible light band. In the terahertz band, the characteristic size of the diffraction element is relatively large (usually in the centimeter range or larger), so manufacturing and experimental debugging are relatively simple. Both 3D printing and traditional processing technologies can be used to fabricate the diffraction structure in the terahertz band. However, due to the large device size, it is difficult to achieve integration, the information processing density is low, and because it is susceptible to interference from other factors in free-space experiments, it is sensitive to the working environment. In the latter visible light band, the diffraction element has a small structural size (micrometer or nanometer scale), high optical field resolution, can perform complex neural network operations in a smaller space for high-density information processing, and can also be integrated with other optical devices to achieve a compact optical neural network chip design. It has great application potential in the fields of computational imaging, image recognition, holographic imaging, etc.
[0038] Currently, the implementation technologies of optical neural networks in the optical band mainly include various solutions such as two-photon polymerization 3D printing, electron beam lithography, nanoimprinting, and liquid crystal spatial light modulators (SLMs). Due to the multiple challenges faced in manufacturing multi-layer optical neural networks, such as high-precision alignment and nano-scale processing difficulties, most of the existing processing technologies cannot simultaneously implement multi-layer (more than 3 layers) and large-area optical neural networks on monolithic integrated devices, making it difficult to realize complex neural networks with both large network depth and width, which limits the realization of their complex functions and their promotion in practical applications. In addition, in the existing optical modulation technologies, it is often difficult to accurately shape the required light field distribution when modulating complex light fields. At the same time, in the light field recognition link, signal processing and feature extraction are relatively difficult, reducing the recognition accuracy.
[0039] In view of this, the present application proposes a method for determining an optical neural network architecture. By iteratively adjusting the optical element parameters of the optical modulation layer according to the output error between the outgoing light field after passing through the optical modulation layer and / or the diffraction layer and the target light field until the output error meets the error preset condition, the output error of the optical neural network can be reduced, thereby improving the recognition accuracy of the optical neural network.
[0040] To facilitate the understanding of this embodiment, first, a method for determining an optical neural network architecture disclosed in the embodiments of the present application will be introduced in detail. The implementation process of the method for determining the optical neural network architecture will be described in detail through several embodiments below.
[0041] Please refer to Figure 1 , which is a flowchart of the method for determining the optical neural network architecture provided by the embodiments of the present application. The following will elaborate on the Figure 1 specific process shown in detail.
[0042] Step S201, determine the outgoing light field after the incident light field passes through the optical modulation layer and / or the diffraction layer in the optical neural network.
[0043] Among them, the optical neural network includes multiple structural layers, and each structural layer includes a corresponding optical modulation layer and / or diffraction layer. The various layers in the optical neural network are connected through diffraction theory.
[0044] The optical modulation layer here is a hierarchical structure that uses optical devices to modulate optical signals. Its main function is to encode and process information by regulating characteristic quantities such as the amplitude and phase of light. By modulating the amplitude or phase distribution of the optical modulation layer, the weight distribution of the optical signal during propagation can be dynamically changed.
[0045] Among them, each layer of the optical modulation layer contains millions of diffraction neurons with trainable and optimized complex amplitude modulation, and the diffraction neurons of each layer serve as point sources of secondary waves connecting the diffraction neurons of the next layer.
[0046] The diffraction layer is the core component that realizes information processing by means of the diffraction principle of light. It is used to connect different optical modulation layers and complete the functions similar to the connection layer and non-linear operation in the artificial neural network.
[0047] It can be understood that after the light passes through the optical modulation layer with a specific microstructure, it undergoes diffraction through the diffraction layer, and the light waves at different positions interfere with each other to form a specific distribution of the output light field.
[0048] Among them, the diffraction calculation process can be realized by algorithms such as Fraunhofer diffraction, Fresnel diffraction or Rayleigh-Sommerfeld diffraction.
[0049] After the incident light field enters the optical neural network, the incident light field propagates through diffraction and enters the optical modulation layer in each structure layer for modulation respectively, and then the light field output by this structure layer is obtained. The light field output by this structure layer serves as the input light field of the next structure layer and continues to be modulated by the optical modulation layer in the next structure layer until the incident light field passes through the last structure layer, completing the modulation of the incident light field and obtaining the output light field.
[0050] Step S202, calculate the output error between the output light field and the target light field.
[0051] The target light field here refers to the output light field that the incident light field passes through the optical neural network in the theoretical case.
[0052] It should be understood that due to the machining error in the machining process, the error caused by the material influence of the structure layer in the optical neural network, etc., after the incident light field passes through the optical neural network, its corresponding output light field may have a certain difference from the target light field. By calculating the output error between the output light field and the target light field and adjusting the optical element parameters of the optical modulation layer based on the output error, the gap between the actual output light field and the target light field can be reduced as much as possible, and then the error value of the optical neural network can be reduced.
[0053] The above output error can be calculated by a loss function. For example, mean square error, cross entropy loss, etc. The determination method of this output error can be selected according to the actual situation.
[0054] In one embodiment, the output error can be determined by the following formula:
[0055]
[0056] Among them, L is the output error, y i is the target light field, is the output light field, N is the number of structure layers, and i is the i-th structure layer.
[0057] Step S203: Adjust the optical element parameters of the optical modulation layer according to the output error.
[0058] Among them, the optical element parameters may include phase distribution, amplitude distribution, interlayer distance, and the corresponding energy distribution and polarization distribution of the processing laser, etc. The optical element parameters can be selected according to the actual situation.
[0059] The optical element parameters here can be determined by an optimization algorithm. For example, gradient descent method, Adam, etc. The determination method of the optical element parameters can be selected according to the actual situation.
[0060] Step S204: Determine the output light field after the incident light field passes through the optical modulation layer and / or diffraction layer after adjusting the optical element parameters, and determine the output error between the output light field and the target light field until the output error meets the preset conditions.
[0061] Step S205: Determine the optical element parameters corresponding to the output error that meets the preset conditions as the target optical element parameters of the optical modulation layer.
[0062] The preset conditions here can be that the error value of the output error meets the preset error range, or it can be to reach the preset number of iterations. The preset conditions can be selected according to the actual situation.
[0063] It should be understood that as Figure 2 shown, by continuously iterating and updating according to the above steps S201 - S203, the error value of the output error can be gradually reduced, and then the output light field output by the optical neural network can continuously approach the preset light field. When the output error meets the preset conditions, it can be determined that the algorithm has converged, and then the optical element parameters of the optimized optical modulation layer are output, and the optical element parameters corresponding to the output error that meets the preset conditions are used as the target optical element parameters of the optical modulation layer to prepare the optical neural network according to the target optical element parameters.
[0064] In the above implementation process, by iteratively adjusting the optical element parameters of the optical modulation layer according to the output error between the output light field after passing through the optical modulation layer and / or diffraction layer and the target light field until the output error meets the error preset conditions, the output error of the optical neural network can be reduced, and then the recognition accuracy of the optical neural network can be improved.
[0065] In a possible implementation manner, determining the output light field after the incident light field passes through the optical modulation layer and / or diffraction layer after adjusting the optical element parameters includes: calculating the intermediate light field after the incident light field passes through the optical modulation layer and / or diffraction layer in each structural layer; determining the intermediate light field output by the last structural layer as the output light field.
[0066] The intermediate light field here can be calculated by the Rayleigh - Sommerfeld diffraction theory.
[0067] It should be understood that when calculating the intermediate optical field of the output of the first structural layer of the computational optical neural network, the incident optical field of the optical neural network can be used as the incident optical field of the first structure. When calculating the intermediate optical field of the output of other structural layers of the optical neural network, the intermediate optical field output by the previous structural layer can be used as the incident optical field of the current layer structure. After calculating the intermediate optical field of the output of the last structural layer of the optical neural network, the intermediate optical field output by the last structural layer is used as the output optical field of the optical neural network.
[0068] In the above implementation process, when calculating the output optical field, specific calculations are performed on the optical fields output by each structural layer respectively, which can improve the accuracy of the output optical field calculation.
[0069] In a possible implementation manner, calculating the intermediate optical field after the incident optical field passes through the optical modulation layer and / or diffraction layer in each structural layer includes: for the first structural layer, determining the optical field modulated by the optical modulation layer in the first structural layer as the intermediate optical field output by the first structural layer; for other structural layers, determining the intermediate optical field output by the current structural layer through the first optical field transmitted through the diffraction layer in the previous structural layer and the second optical field modulated by the optical modulation layer in the current structural layer.
[0070] Among them, in the optical neural network, the intermediate optical field output by the previous structural layer serves as the input optical field of the next structure.
[0071] The propagation of the above incident optical field in the optical neural network can be calculated by the following formula:
[0072] The propagation of the optical field can be calculated by the following formula:
[0073] U(x,y) = ∫∫H(x,y; x′,y′,z)U(x′,y′,0)dx′dy′;
[0074] Among them, U(x,y) is the optical field after diffraction, H(x,y; x′,y′,z) is the diffraction transfer function, and U(x′,y′,0) is the optical field modulated by the upper optical modulation layer.
[0075] The diffracted optical field U(x,y) obtained through diffraction calculation is modulated by the next optical modulation layer, and the modulation matrix M(x,y) of the optical modulation layer can be expressed as:
[0076]
[0077] Among them, M(x,y) is the modulation matrix, a(x,y) is the modulation amplitude, is the modulation phase.
[0078] The optical field U(x′, y′, 0) modulated by the upper optical modulation layer, after diffraction propagation and complex amplitude modulation by the optical modulation layer of this layer, the intermediate optical field U out (x, y) can be expressed as:
[0079] U out (x, y) = M(x, y) · ∫∫H(x, y; x′, y′, z)U(x′, y′, 0)dx′dy′;
[0080] Among them, U out (x, y) is the intermediate optical field, M(x, y) is the modulation matrix, H(x, y; x′, y′, z) is the diffraction transfer function, and U(x′, y′, 0) is the optical field modulated by the upper optical modulation layer.
[0081] It should be understood that when calculating the intermediate optical field of the output of the first structural layer of the optical neural network, U(x′, y′, 0) is the incident optical field.
[0082] In the above implementation process, when calculating the output optical field, the intermediate optical field after the incident optical field passes through the optical modulation layer and the diffraction layer in each structural layer is calculated respectively, and then the intermediate optical field output by the previous structural layer is used as the input optical field of the next structural layer. Specific calculations are performed on the optical fields output by each structural layer, which can realize the accurate recognition and modulation of complex incident optical fields, thereby improving the accuracy of the calculation of the output optical field.
[0083] In a possible implementation manner, step S203 includes: calculating the gradient of the loss function with respect to the optical element parameters in each structural layer through the chain rule; determining the contribution value of each structural layer to the output optical field through the backpropagation algorithm; adjusting the optical element parameters of the optical modulation layer in each structural layer according to the optimization algorithm and the contribution value of each structural layer to the output optical field.
[0084] Among them, the loss function is used to calculate the output error between the output optical field and the target optical field.
[0085] In the process of adjusting and optimizing the optical element parameters, the value of the complex amplitude modulation can be adjusted by error backpropagation. In the error backpropagation process, the gradient of the loss function with respect to the optical element parameters of each layer is calculated through the chain rule.
[0086] The contribution value here can be calculated through the following formula:
[0087]
[0088] Among them, w j is the parameter of the j-th layer, and z j is the output of the j-th layer.
[0089] The above optimization algorithms can be gradient descent method, Adam, etc., and the optimization algorithm can be selected according to the actual situation.
[0090] In one embodiment, the optical element parameters can be adjusted by the following formula:
[0091]
[0092] where η is the learning rate, is the adjusted optical element parameter, is the optical element parameter before adjustment, is the contribution value.
[0093] In the above implementation process, by adjusting the optical element parameters according to the output error, the accuracy of the optical element parameters can be improved. In addition, the optical key parameter adjustment method in the above embodiment is a training method designed specifically for a monolithic multi-layer large-area optical neural network. At the same time, considering compensating for the manufacturing output error, the robustness of the system to manufacturing defects can be improved through structural redundancy and / or algorithm optimization, and the accuracy of the optical neural network architecture can be improved.
[0094] In a possible implementation manner, calculating the intermediate light field after the incident light field passes through the optical modulation layer and / or diffraction layer in each structural layer includes: incorporating the actual manufacturing error distribution into the diffraction theory and performing parameter optimization; calculating the intermediate light field after the incident light field passes through the optical modulation layer and / or diffraction layer in each structural layer according to the diffraction theory.
[0095] The actual manufacturing error here refers to the error that may occur during the actual manufacturing process of the optical neural network. For example, alignment error, structural error, distance error, etc. The actual manufacturing error can be selected according to the actual situation.
[0096] It should be understood that during the actual manufacturing process of the optical neural network, it is usually difficult to manufacture it accurately according to the designed neural network architecture. In fact, due to limitations such as process, material, and process, manufacturing errors are usually introduced when manufacturing the optical neural network. In order to avoid the problem that there is still a large error between the output light field and the target light field of the manufactured optical neural network due to manufacturing errors, resulting in low accuracy of the optical neural network. During the process of determining the optical neural network architecture, the actual manufacturing error is incorporated into the diffraction theory to consider the influence of the actual error on the optical neural network, so that the determined optical neural network architecture will not have a great impact on the accuracy of the light field output by the optical neural network architecture even if there are manufacturing errors during the manufacturing process.
[0097] The actual manufacturing error here can be obtained through multiple experiments or determined according to a set model, and the actual manufacturing error can be selected according to the actual situation.
[0098] In one embodiment, the actual fabrication error can be introduced each time model training is performed. That is, an offset corresponding to the actual fabrication error is introduced during the diffraction distance training process.
[0099] In the above implementation process, during the process of parameter optimization, introducing the actual fabrication error can obtain an optical neural network architecture that takes into account the actual fabrication error, avoiding the problem of low accuracy of the fabricated optical neural network caused by the introduction of errors during the actual fabrication process of the optical neural network, improving the model's tolerance to errors, and thereby improving the accuracy of the optical neural network.
[0100] Please refer to Figure 3 , which is a flowchart of the method for preparing an optical neural network provided by an embodiment of the present application. The following will elaborate in detail on Figure 3 the specific process shown.
[0101] Step S301, determine the processing parameters for fabricating the optical neural network according to the target optical element parameters determined in the above embodiment.
[0102] Among them, the target optical element parameters include: working wavelength, interlayer distance, phase distribution, and / or amplitude distribution; the processing parameters include: processing energy, processing polarization, and processing position.
[0103] Parameters such as the working wavelength, phase distribution, and amplitude distribution here can be used to determine processing parameters such as processing energy and processing polarization. The interlayer distance can be used to determine processing parameters such as the processing position.
[0104] It should be understood that after determining the optical neural network architecture, the target optical element parameters of each optical element in the optical neural network to be fabricated are obtained, and then the corresponding processing parameters can be determined based on the target optical element parameters of the optical element, and then the optical neural network is fabricated through the processing parameters.
[0105] Step S302, fabricate the optical neural network inside the medium through the processing parameters.
[0106] The medium here is a single-piece medium.
[0107] In one embodiment, the medium is a transparent medium.
[0108] It should be understood that after determining the processing parameters, a multi-layer phase or complex amplitude modulation device is fabricated inside the single-piece transparent medium according to the processing parameters, without subsequent high-precision alignment, and has a high integration degree.
[0109] In the above implementation process, by determining the processing parameters according to the target optical element parameters, the accuracy of the processing parameters can be improved. In addition, when processing the optical neural network in a single piece of medium, subsequent high-precision alignment is not required, and the integration level is high, which can reduce the processing difficulty of the optical neural network and improve the processing efficiency and accuracy.
[0110] In a possible implementation manner, step S301 includes: determining the processing energy and processing polarization according to the phase distribution and / or amplitude distribution, and the working wavelength; determining the processing position according to the interlayer distance.
[0111] It should be understood that after determining the target optical element parameters such as the phase distribution, amplitude distribution, and working wavelength of each optical element in the optical neural network, in order to enable the phase distribution, amplitude distribution, and working wavelength of each optical element in the processed optical neural network to reach the phase distribution, amplitude distribution, and working wavelength corresponding to the optical neural network architecture, determining the corresponding processing energy and processing polarization through the phase distribution, amplitude distribution, and working wavelength can enable the laser to process the optical neural network according to the phase distribution, amplitude distribution, and working wavelength corresponding to each optical element, thereby improving the processing accuracy.
[0112] In addition, after determining the interlayer distance between each structural layer in the optical neural network and determining the processing position of the laser during the processing according to the interlayer distance, and then controlling the laser to be located at the corresponding position when processing each structural layer, so that the interlayer distance of the processed structural layer is the interlayer distance corresponding to the optical neural network architecture, thereby improving the processing accuracy.
[0113] In the above implementation process, by determining the processing energy and processing polarization according to the phase distribution and / or amplitude distribution, and the working wavelength, and determining the processing position according to the interlayer distance, the laser can process the optical neural network according to the corresponding processing parameters, so that the optical element parameters of each optical element of the processed optical neural network are all target optical element parameters, thereby improving the processing accuracy.
[0114] Please refer to Figure 4 , which is a schematic structural diagram of an optical neural network preparation system provided by an embodiment of the present application, including: a laser 100, an energy control optical path 200, a polarization control optical path 300, an objective lens 400, and a moving platform 500.
[0115] Among them, the moving platform 500 is used to place the device to be processed.
[0116] The laser 100 here is used to emit a laser beam, the moving platform 500 is used to adjust the processing position of the device to be processed, the energy control optical path 200 is used to adjust the processing energy of the laser beam, the polarization control optical path 300 is used to adjust the processing polarization of the processing energy in real time, and the objective lens 400 is used to focus the laser beam into the inside of a single medium.
[0117] Among them, the processing position, processing energy, and processing polarization are determined by the target optical element parameters determined by the optical neural network architecture determination method in the above embodiments.
[0118] The energy control optical path 200 here includes a first half-wave plate, a polarization beam splitter prism, and a second half-wave plate. Among them, the first half-wave plate, the polarization beam splitter prism, and the second half-wave plate jointly control the pulse energy.
[0119] The energy control optical path 200 of this can also include an electro-optic modulator. Among them, by applying an electric field to change the refractive index, the intensity of the transmitted light is adjusted.
[0120] The energy control optical path 200 of this can also include a variable optical density filter, an acousto-optic modulator, etc. Among them, by driving the grating diffraction with ultrasonic waves, the intensity of the diffracted light is adjusted, thereby adjusting the laser energy.
[0121] The structure of the above energy control optical path 200 is only exemplary, and the structure of the energy control optical path 200 can be selected according to the actual situation.
[0122] The polarization control optical path 300 here includes an electro-optical modulator. Among them, by applying different voltage values to the electro-optical modulator, real-time changes in the polarization direction can be achieved.
[0123] In one embodiment, after the laser passes through energy control and polarization control, it is split by a beam splitter prism. One beam of light enters the image acquisition device for real-time monitoring of the beam, and the other beam of light is focused into the medium by the objective lens 400.
[0124] The laser focusing position is adjusted by the moving platform 500. During the laser processing, by adjusting the processing parameters and processing position, the processing of any multi-layer structure can be achieved inside the medium.
[0125] Among them, the laser beam processes the input layer, output layer, and structure layer inside the single medium to form an optical neural network.
[0126] The above moving platform 500 is a three-dimensional displacement stage that can be displaced in three mutually perpendicular directions. The device to be processed is arranged on the moving platform 500. When the moving platform 500 moves, it drives the device to be processed to move, thereby realizing the adjustment of the laser focusing position.
[0127] In the above implementation process, by adopting the laser multi-layer processing technology, and the optical modulation layer is processed inside a single piece of medium, that is, multiple optical modulation layers are processed inside the same piece of medium. High-precision alignment is achieved during the processing between layers, and it is more integrated, which can improve the environmental adaptability of the optical neural network. In addition, since the system does not rely on circuit signal processing and its monolithic structure, it can still work normally under strong electromagnetic interference or extreme temperature environments, increasing the application scenarios of the optical neural network preparation system.
[0128] Please refer to Figure 5 , which is a schematic structural diagram of the optical neural network provided by an embodiment of the present application, including: an input layer, an output layer, and a structural layer.
[0129] Among them, the structural layer is arranged between the input layer and the output layer.
[0130] The structural layer here includes one or more layers; each layer of the input layer, the structural layer, and the output layer is connected through the diffraction theory.
[0131] Among them, the structural layer includes an optical modulation layer and / or a diffraction layer, and this structural layer is configured to modulate the incident light field.
[0132] In one embodiment, the optical element parameters of the optical modulation layer in the structural layer are determined by the optical neural network architecture determination method in the above embodiment.
[0133] Optionally, the material of the diffraction layer can be a transparent dielectric material, an optical thin film, a photonic crystal, etc. The material of this diffraction layer can be selected according to the actual situation.
[0134] The above input layer is configured to receive the incident light field, and the output layer is configured to output the light field modulated by the structural layer.
[0135] It should be understood that this optical neural network can accurately control the attributes of the optical signal (such as phase modulation, amplitude modulation, etc.) through multiple structural layers to simulate the calculation process of the neural network. This control method can optimize the parameters of multiple optical modulation layers for multiple targets simultaneously, thereby improving the processing efficiency of the system. It can also enable the device to work effectively at different wavelengths, polarizations, etc., and can be adjusted with different functions according to requirements.
[0136] In addition, this optical neural network is a monolithic integrated structure. By designing the phase amplitude modulation of the metasurface to achieve the neural network calculation function, and using the ultrafast laser multi-layer processing technology to implement it on an optical device, low power consumption, fast processing, and high-parallel computing capabilities can be achieved.
[0137] Optionally, the optical neural network can be applied to fields such as image processing, computer vision tasks, communication, optical sensing, data storage, etc. For example, using the same diffraction calculation and device structure, signal modulation and demodulation can be performed in optical communication, or the diffraction effect can be used to capture and process signals in an optical sensor, etc.
[0138] In the above implementation process, through the multi-layer structure layer, the incident light field can be modulated layer by layer to achieve precise recognition and modulation of the complex incident light field. In addition, the recognition accuracy depends on the number of layers of the optical neural network and the design of the optical modulation layer of each layer, enabling the optical neural network to capture minute feature changes in the input, suitable for various tasks such as image recognition and computational imaging, and increasing the application scenarios of the optical neural network.
[0139] In a possible implementation manner, the input layer, output layer, and structure layer are arranged inside a single-piece medium.
[0140] The input layer, output layer, and structure layer here can be processed inside the medium using ultrafast laser processing technology.
[0141] It should be understood that through ultrafast laser processing technology, large-area manufacturing of optical neural networks can be achieved. Based on the principle of metasurface optical field regulation, the amplitude, phase, or complex amplitude of the optical field can be regulated. The computational ability of large-area networks is stronger, and the degree of freedom of optical field regulation is higher, enabling multiplexing of multiple parameters such as the wavelength and angular momentum of light.
[0142] In an embodiment, the medium is a single-piece transparent medium. By setting the optical neural network in the single-piece transparent medium, the optical neural network has high integration, a stable and efficient structure, and characteristics of broadband, high efficiency, and high laser loss threshold. Moreover, the obtained optical neural network has many layers and a large area, and can realize functions such as complex optical calculations and image recognition processing.
[0143] The optical neural network in the embodiments of the present application can realize various functions such as complex optical calculations, image recognition, and optical field regulation. The following shows specific examples of the use of the optical neural network in the embodiments of the present application through several embodiments:
[0144] Example 1: Input of multi-wavelength light sources: Select two or more light sources with different wavelengths (such as red, green, and blue lasers), and each wavelength light source corresponds to a different input signal. Through a spatial light modulator or other optical devices, the light waves of each wavelength are loaded with the corresponding information, so that they already contain different information to be processed before entering the optical neural network.
[0145] Example 2: Multi-layer structure design: Each diffraction propagation layer is designed for the optical field characteristics of multiple wavelengths, so that light of different wavelengths generates different diffraction patterns in each layer. The multi-layer structure of the optical neural network can modulate different wavelengths simultaneously, enabling the light of each wavelength to propagate out with an independent characteristic light field distribution. For example, red light can be used to recognize digital images, green light to recognize characters, and blue light to detect object contours.
[0146] Example 3: Output hologram with wavelength multiplexing: After multi-layer diffraction propagation, the light of each wavelength forms a specific holographic image in the output layer. By using an image acquisition device or a photodetector to detect the light field distribution of each wavelength respectively, multiple recognition results of different categories can be obtained simultaneously.
[0147] In the above implementation process, by processing multi-layer structural layers inside the same medium, high-precision alignment is achieved during processing between layers, and it is more integrated, improving the environmental adaptability of the optical neural network.
[0148] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that similar reference numerals and letters indicate similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.
[0149] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims
1. A method for determining an optical neural network architecture, characterized in that Including: Determine the output optical field after the incident optical field passes through the optical modulation layer and / or diffraction layer in the optical neural network; Calculate the output error between the output optical field and the target optical field; Adjust the optical element parameters of the optical modulation layer according to the output error; Determine the output optical field after the incident optical field passes through the optical modulation layer and / or the diffraction layer with adjusted optical element parameters, and determine the output error between the output optical field and the target optical field until the output error meets the preset conditions; Determine that the optical element parameters corresponding to the output error that meets the preset conditions are the target optical element parameters of the optical modulation layer.
2. The method according to claim 1, wherein Wherein, The optical neural network includes multiple structural layers, and each structural layer includes a corresponding optical modulation layer and / or diffraction layer; The determining the output optical field after the incident optical field passes through the optical modulation layer and / or diffraction layer in the optical neural network includes: Calculate the intermediate optical field after the incident optical field passes through the optical modulation layer and / or the diffraction layer in each structural layer; Determine the intermediate optical field output by the last structural layer as the output optical field.
3. The method according to claim 2, wherein The calculating the intermediate optical field after the incident optical field passes through the optical modulation layer and / or the diffraction layer in each structural layer includes: For the first structural layer, determine the optical field modulated by the optical modulation layer in the first structural layer as the intermediate optical field output by the first structural layer; For other structural layers, determine the intermediate optical field output by this structural layer through the first optical field transmitted through the diffraction layer in the previous structural layer and the second optical field modulated by the optical modulation layer in this structural layer; Wherein, in the optical neural network, the intermediate optical field output by the previous structural layer serves as the input optical field of the next structural layer.
4. The method according to any one of claims 1 to 3, characterized in that, The adjusting the optical element parameters of the optical modulation layer according to the output error includes: Calculate the gradient of the loss function with respect to the optical element parameters in each structural layer through the chain rule; wherein, the loss function is used to calculate the output error between the output optical field and the target optical field; Determine the contribution value of each structural layer to the output optical field through the backpropagation algorithm; Adjust the optical element parameters of the optical modulation layer in each structural layer according to the optimization algorithm and the contribution value of each structural layer to the output optical field.
5. The method according to any one of claims 1 to 3, characterized in that The calculating the intermediate optical field after the incident optical field passes through the optical modulation layer and / or the diffraction layer in each structural layer includes: Incorporate the actual fabrication error distribution into the diffraction theory and perform parameter optimization; Calculate the intermediate optical field after the incident optical field passes through the optical modulation layer and / or the diffraction layer in each structural layer according to the diffraction theory.
6. A method for preparing an optical neural network, characterized in that, Including: Determine the processing parameters for fabricating the optical neural network according to the target optical element parameters determined by the method according to any one of claims 1-5; Fabricate the optical neural network inside the medium through the processing parameters.
7. The method according to claim 6, wherein Wherein, The target optical element parameters include: working wavelength, interlayer distance, phase distribution, and / or amplitude distribution; the processing parameters include: processing energy, processing polarization, and processing position. Determine the processing parameters for machining the optical neural network based on the target optical element parameters determined by the method according to any one of claims 1-5, including: Determine the processing energy and processing polarization according to the phase distribution and / or the amplitude distribution, and the working wavelength. Determine the processing position according to the interlayer distance.
8. An optical neural network preparation system, characterized in that, Including: A laser, an energy control optical path, a polarization control optical path, an objective lens, and a moving platform; The moving platform is used to place the device to be processed; The laser is used to emit a laser beam; The moving platform is used to adjust the processing position of the device to be processed; The energy control optical path is used to adjust the processing energy of the laser beam; The polarization control optical path is used to adjust the processing polarization of the processing energy in real time; wherein, the processing position, the processing energy, and the processing polarization are determined by the target optical element parameters determined by the method according to any one of claims 1-5; The objective lens is used to focus the laser beam into the single-piece medium; Wherein, the laser beam performs input layer, output layer, and structure layer processing inside the single-piece medium to form an optical neural network.
9. An optical neural network, characterized in that, Including: An input layer, an output layer, and a structure layer; The structure layer is arranged between the input layer and the output layer; wherein, the structure layer includes one or more layers; each layer of the input layer, the structure layer, and the output layer is connected through diffraction theory; The input layer is configured to receive an incident light field; The structure layer is configured to modulate the incident light field; The output layer is configured to output the light field modulated by the structure layer; Wherein, the optical element parameters of the optical modulation layer in the structure layer are determined by the method according to any one of claims 1-5.
10. The optical neural network according to claim 9, characterized in that, Wherein, The input layer, the output layer, and the structure layer are arranged inside a single-piece medium.