Optical neural network based on width learning theory

Abstract

The invention discloses an optical neural network based on a width learning theory, which comprises a feature modulation module, an enhancement modulation module, a combination connection module and an optical classification module which are connected in sequence, and is characterized in that the feature modulation module is used for carrying out feature modulation processing on an input light field image to generate a parallel feature node image; the enhanced modulation module is used for performing enhanced modulation processing on the input parallel feature node image to generate a plurality of enhanced node images subjected to nonlinear optical modulation; the combined connection module is used for performing combined processing on the input parallel feature node image and the enhanced node image to generate a combined node image; the optical classification module is used for optically classifying the input combined node image to generate a light output vector. A design principle of a width learning theory is utilized to convert an optical computing system from a vertical cascade structure to a width-based parallel structure; under a shallow and wide structure, the problem of error accumulation in optical mapping is effectively solved.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to an optical neural network based on width learning theory, which can be used for image processing tasks such as image classification or image reconstruction. Background Art

[0002] In recent years, there have been an increasing number of big data applications across various fields, including medicine, industry, and art. Correspondingly, as we approach the post-Moore's Law era, the demand for computing power is also rising. Traditional silicon-based electronic devices using von Neumann architecture neural networks face physical limitations such as processing speed, energy consumption, and bandwidth, which have led to bottlenecks in computer progress. To address these challenges, optical computing has been widely studied, and researchers have used its fast transmission speed, strong information carrying capacity, high parallelism, and robustness to interference to overcome the limitations faced by electronic neural networks based on von Neumann architecture.

[0003] An optical neural network system is a system that uses light as a medium to transmit information. It uses phase, amplitude or frequency modulation to load and process signals. Compared with traditional silicon-based electronic components, optical computing has higher parallelism and electromagnetic radiation interference. Current research on optical computing systems focuses on two architectures: an integrated on-chip optical neural network architecture for building general optical computing systems, and a spatial optical neural network architecture tailored for specific tasks.

[0004] However, the composition of traditional optical computing systems is based on the physical system of optical devices, which fundamentally involves manipulating the image light field in the optical field to achieve intelligent information extraction; therefore, the system is highly sensitive to the alignment errors and manufacturing inaccuracies of individual components. The current design paradigm of spatial optical neural network architecture aims to improve the performance of optical neural network systems by cascading optical devices, thereby increasing the depth of optical systems; however, the deep cascading of traditional networks easily introduces errors at each layer, and there is a problem of light field energy loss, which impairs the ability of optical neural network systems; therefore, traditional optical neural network architectures are difficult to construct deep cascade structures, which is one of the bottlenecks in the current development of optical neural network systems. Summary of the invention

[0005] In view of the drawback that the existing cascaded optical system is prone to introduce errors, the present invention proposes an optical neural network based on width learning theory. By utilizing the design principles of width learning theory, the optical computing system is converted from a vertical cascade structure to a width-based parallel structure. Under the shallow-wide structure, the error accumulation problem in cascaded optical mapping is effectively solved.

[0006] To achieve the above tasks, the present invention adopts the following technical solutions:

[0007] An optical neural network based on width learning theory comprises a feature modulation module, an enhancement modulation module, a combination connection module and an optical classification module connected in sequence, wherein:

[0008] The feature modulation module is used to perform feature modulation processing on the input light field image to generate parallel feature node images; during the feature modulation process, the light field image is copied into multiple light field image copies, and modulated with a spatial light modulation matrix of the same size as the light field image, so as to achieve the output of multiple parallel feature node images of the same size and different modulations;

[0009] The enhanced modulation module is used to perform enhanced modulation processing on the input parallel feature node images to generate multiple enhanced node images that have undergone nonlinear optical modulation; before the enhanced modulation process, the parallel feature node images are first subjected to beam separation to obtain two sets of identical parallel feature node images; during the enhanced modulation process, one set of parallel feature node images enters the combined connection module, and the other set of parallel feature node images uses the optical nonlinear layer to perform nonlinear modulation of the image, thereby achieving the output of multiple enhanced node images of the same size;

[0010] The combination connection module is used to combine the input parallel feature node image and the enhanced node image to generate a combined node image; during the combination connection process, the two beams of the parallel feature node image and the enhanced node image are distributed in parallel, thereby realizing the output of the combined node image;

[0011] The optical classification module is used to perform optical classification on the input combined node image to generate a light output vector; during the optical classification process, the combined node image is processed by optical linear matrix multiplication to generate a light output vector.

[0012] Furthermore, the feature modulation module includes a digital micromirror device unit, a microlens array unit, and a spatial light modulator unit, wherein:

[0013] The digital micromirror device unit uses an incoherent broadband illumination field to perform image loading processing on an input optical image to obtain an input data field;

[0014] The microlens array unit adopts optical fan-out to generate multiple parallel optical image copies of the input data field to obtain a parallel input data field;

[0015] The spatial light modulator unit adopts a spatial light modulation matrix to perform matrix weighted modulation on each optical image copy in the parallel input data field to obtain parallel characteristic node images.

[0016] Further, the enhanced modulation module includes a beam splitter unit and an optical nonlinear layer unit, wherein:

[0017] The beam splitter unit adopts an optical beam splitter for splitting the light field transmission beam of the parallel characteristic node image into two identical beams, thereby obtaining two sets of identical parallel characteristic node images;

[0018] The optical nonlinear layer unit specifically utilizes the nonlinear photoelectric effect to perform nonlinear image modulation processing on a set of parallel characteristic node images input to obtain enhanced node images.

[0019] Furthermore, the optical nonlinear layer does not change the shape or size of the light beam, but changes the intensity of the light to varying degrees, thereby achieving nonlinear adjustment.

[0020] Furthermore, the combined connection module includes a relay lens unit, wherein:

[0021] The relay lens unit adopts relay lenses with different focal lengths, does not change the size of the image or the information content it carries, and performs imaging at different positions to achieve optical shaping; the two beams of the parallel feature node image and the enhanced node image are spliced ​​together in parallel at the same place using the relay lens unit to obtain a combined node image.

[0022] Further, the optical classification module includes a microlens array unit, a spatial light modulator unit, and a lens unit, wherein:

[0023] The microlens array unit adopts optical fan-out to generate a corresponding input light field for the combined node image;

[0024] The spatial light modulator unit uses a spatial light modulator of matrix multiplication to linearly modulate the input light field to obtain a light field output image equal to the number of task categories;

[0025] The lens unit adopts an optical lens to converge the light field output image to obtain the final light output vector.

[0026] Furthermore, the training of the optical neural network based on the width learning theory adopts a back propagation algorithm;

[0027] After the optical image to be processed is obtained, it is input into the trained network model to obtain the corresponding light output vector.

[0028] Furthermore, the optical output vector is used for image classification or image reconstruction.

[0029] An optical image processing unit is provided with a trained optical neural network model based on width learning theory for classifying or reconstructing an input optical image.

[0030] A computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the optical neural network based on width learning theory realizes the classification or reconstruction process of the optical image.

[0031] Compared with the prior art, the present invention has the following technical features:

[0032] 1. The present invention uses a shallow width optical diffraction structure to construct a spatial optical neural network architecture, which is an important breakthrough in this field. It perfectly avoids the inherent optical system error problems associated with deep cascades in traditional optical neural network systems, while ensuring the same performance as the baseline cascade optical neural network system.

[0033] 2. Compared with the traditional optical computing system, the optical neural network constructed by the method of the present invention has a significant improvement in convergence speed, which is nearly two orders of magnitude. It can also achieve parameter reduction without reducing accuracy, thereby making the calculation more efficient; in addition, the test of misalignment error proves that compared with the traditional optical neural network system, the new spatial optical neural network structure has sufficient and superior noise resistance. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the principle of the optical neural network provided by the present invention;

[0035] Figure 2 It is a schematic diagram of inputting into the characteristic node calculation in the figure of the present invention;

[0036] Figure 3 It is a schematic diagram of the calculation from the characteristic node to the combination node in the figure of the present invention;

[0037] Figure 4 It is a schematic diagram of combining nodes to output calculation in the present invention;

[0038] Figure 5 It is the convergence curve diagram of the network on the two data sets used for simulation experiments, where (a) is the convergence curve diagram of the Qcuikdraw data set, and (b) is the convergence curve diagram of the Fashion-MNIST data set;

[0039] Figure 6 These are the influence diagrams of the error on the network on two sets of data used for simulation experiments, where (a) is the influence diagram of the Qcuikdraw dataset, and (b) is the influence diagram of the Fashion-MNIST dataset;

[0040] Figure 7 It is a schematic diagram of the reconstruction task using the existing method and the present invention, wherein (a) is the original image, (b) is the reconstruction result of the traditional MLP-3 network, and (c) is the reconstruction result of the network of the present invention. DETAILED DESCRIPTION

[0041] Referring to the accompanying drawings, the present invention provides an optical neural network based on width learning theory, comprising a feature modulation module, an enhancement modulation module, a combination connection module and an optical classification module connected in sequence, wherein:

[0042] 1. Feature modulation module

[0043] The feature modulation module is used to perform feature modulation processing on the input light field image to generate parallel feature node images; during the feature modulation process, the light field image is copied into multiple light field image copies and modulated with a spatial light modulation matrix of the same size as the light field image, thereby realizing the output of multiple parallel feature node images of the same size but under different modulations.

[0044] A characteristic modulation module provided in one embodiment of the present invention is as follows Figure 2 As shown, the feature modulation module includes a digital micromirror device (DMD) unit, a microlens array (MLA) unit, and a spatial light modulator (LCD) unit, wherein:

[0045] The DMD unit specifically uses an incoherent broadband illumination field to perform image loading processing on an input optical image to obtain an input data field;

[0046] The MLA unit uses optical fan-out to generate multiple parallel optical image copies of the input data field to obtain parallel input data fields;

[0047] The LCD unit adopts a spatial light modulation matrix to perform matrix weighted modulation on each optical image copy in the parallel input data field to obtain parallel characteristic node images.

[0048] For the input image data X, the output of the feature modulation module is:

[0049]

[0050] Among them, Z n is the nth set of feature node images generated, N is the number of optical image copies, is the optical mapping weight of the nth group, β n is the deviation; then, these generated mapping feature groups are collected together as parallel feature node images Z = [Z1, Z2, ..., Z N ].

[0051] Traditional width learning theory cannot be directly applied to the training of optical neural networks. Specifically, linear modulation is transformed from matrix-vector multiplication in width learning to matrix-dot multiplication. In optical neural networks, matrix-vector multiplication is not easy to implement and requires a large amount of optical equipment, while matrix multiplication only requires the light beam to carry input information through the LCD to load the modulation matrix. The inventors have verified that the network construction can still maintain high accuracy using the dot product method.

[0052] Among them, there are constraints on the weights of the spatial light modulation matrix; for the design of the optical neural network, the weights are limited to the range (0, 1), which complies with the rules of transmission intensity modulation; after passing through the spatial light modulation matrix, the intensity of the light beam decreases or remains unchanged; this is reflected in the training process because the weights of the modulation matrix are limited to the range (0, 1).

[0053] In the design of the feature modulation module of the present invention, the main factor considered is to generate multiple input data fields in optical parallel. The purpose of this design is to obtain multiple linear modulation images using different modulation matrices in this scheme, so as to make the features of the optical linear extraction of a single layer more diverse. Among them, the feature modulation mode can select different optical modulation modes, and this scheme is single-layer optical linear modulation.

[0054] After the feature modulation module, the parallel feature node images are used in the subsequent enhanced modulation module and combined connection module respectively.

[0055] 2. Enhanced modulation module

[0056] The enhanced modulation module is used to perform enhanced modulation processing on the input parallel feature node images to generate multiple enhanced node images that have undergone nonlinear optical modulation; before the enhanced modulation process, the parallel feature node images are first subjected to beam separation to obtain two sets of identical replicated parallel feature node images; during the enhanced modulation process, one set of parallel feature node images enters the combined connection module, and the other set of parallel feature node images uses the optical nonlinear layer to perform nonlinear modulation of the image, and the nonlinear modulation does not change the image size, thereby achieving the output of multiple enhanced node images of the same size.

[0057] The enhanced modulation module provided by the present invention is as follows Figure 3 As shown in the upper part after the beam is separated, the enhanced modulation module includes a beam splitter unit and an optical nonlinear layer unit, wherein:

[0058] The beam splitter unit adopts an optical beam splitter for splitting the light field transmission beam of the parallel characteristic node image into two identical beams, thereby obtaining two sets of identical parallel characteristic node images;

[0059] The optical nonlinear layer unit specifically utilizes the nonlinear photoelectric effect to perform image nonlinear modulation processing on a set of parallel characteristic node images input to obtain an enhanced node image; the optical nonlinear layer does not change the shape or size of the light beam, but changes the intensity of the light to varying degrees, thereby achieving nonlinear adjustment;

[0060] For the parallel feature node image Z, the calculation of the enhanced modulation module is based on the secondary optical mapping of the parallel feature node image Z, which can be expressed as:

[0061] H=ζ(ZW H +β H )

[0062] Among them, W H is the secondary optical mapping weight, β H is the enhancement bias, ζ(·) is the optical nonlinear activation function, and H is the enhanced node image.

[0063] Among them, the setting of the optical nonlinear layer unit needs to satisfy: when the input is 0, the output should also be 0, and both the input and output should be positive. In addition, several conditions need to be met to realize optical nonlinearity in the experiment: for example, it has a sufficiently fast response time. Since light propagates very fast in optical elements, nonlinear effects usually become significant when the intensity of the light field changes rapidly. It is best to have a certain gain and reduce energy consumption. Since optical devices such as DMD and LCD exhibit significant energy loss when light propagates through them, gain can be used to amplify weak light signals, which is conducive to the realization of multi-layer optical neural networks. In addition, the energy consumption of the entire network is determined by nonlinearity, so low energy consumption can further utilize the advantages of optical neural networks.

[0064] Among them, before the nonlinear processing is performed by the optical nonlinear layer, the enhancement node can increase the weight and bias through a specific optical device, such as an optical matrix vector multiplier. Multiple modulations through the enhancement node are conducive to completing complex tasks.

[0065] In the design of the enhanced modulation module of the present invention, the main factor considered is the further optical modulation of the feature node image. The purpose of such design is to produce higher-order abstract light field image feature results under the coupling of complex optical modulation, thereby improving the feature representation ability of the optical neural network. Among them, the enhanced modulation method can increase the weights and deviations through specific optical devices such as optical matrix vector multipliers, thereby obtaining multiple modulations of the enhanced nodes, which is conducive to completing complex tasks. This scheme is a single-layer optical nonlinear modulation.

[0066] After the enhancement adjustment module, the enhanced node image is used to be optically connected in parallel with the parallel feature node image in the subsequent combination connection module and output.

[0067] 3. Combined connection module

[0068] The combined connection module is used to combine the input parallel feature node image and enhanced node image to generate a combined node image; during the combined connection process, the two light beams of the parallel feature node image and the enhanced node image are distributed in parallel, thereby realizing the output of the combined node image.

[0069] The combined connection module provided by the present invention is as follows Figure 3 As shown in the beam combination, the combined connection module includes a relay lens unit, wherein:

[0070] The relay lens unit uses relay lenses with different focal lengths, which will not change the size of the image or the information content it carries; on the contrary, its purpose is to perform imaging at different positions to achieve optical shaping to meet the requirements of the same optical path length; the two beams of the parallel feature node image and the enhanced node image are spliced ​​together in parallel at the same place using the relay lens unit to obtain a combined node image.

[0071] For the characteristic node image Z and the enhanced node image H, the calculation of the combined connection module is based on the combination of the characteristic nodes and the optical nodes to obtain the combined node image Z_H; it can be expressed as:

[0072] Z_H=[Z|H]

[0073] The combination realizes node alignment through optical shaping, and [Z|H] represents the splicing and combination operation of Z and H.

[0074] In the design of the combined connection module of the present invention, the main factor taken into consideration is the combination of the parallel feature node image and the enhanced node image. The purpose of such a design is to combine the optical result advantages of the parallel feature node image under simple optical modulation and the enhanced node image under complex optical modulation, thereby constructing a single-layer optical network layer with a high feature resolution capability.

[0075] After the combined connection module, the combined node image is used in the subsequent optical classification module.

[0076] 4. Optical classification module

[0077] The optical classification module is used to perform optical classification on the input combined node image to generate a light output vector; wherein during the optical classification process, the combined node image is processed by optical linear matrix multiplication to obtain the light output vector.

[0078] The optical classification module provided by the present invention is as follows Figure 4 As shown, the optical classification module includes a microlens array (MLA) unit, a spatial light modulator (LCD) unit, and a lens (LEN) unit, wherein:

[0079] The MLA unit uses optical fan-out to generate the corresponding input light field for the combined node image;

[0080] The LCD unit uses a spatial light modulator with matrix multiplication to linearly modulate the input light field to obtain a light field output image equal to the number of task categories;

[0081] The LEN unit uses an optical lens to converge the light field output image to obtain the final light output vector; the optical output vector can be used for image classification or further processed by a digital processing module for image reconstruction.

[0082] For the combined node image Z_H, the light output vector Y of the optical neural network system based on the width learning theory can be calculated as:

[0083] Y=Z_H·W F

[0084] =[Z|H]W F

[0085] Among them, W F is the final output weight.

[0086] In the design of optical classification of the present invention, the main factor considered is to match the combined node image result with the task. The purpose of such design is to obtain the corresponding light output vector, which can be used for subsequent specific optical task requirements.

[0087] Based on the above four modules, the present invention constructs a new optical neural network architecture with shallow width. Different from the traditional optical neural network that extracts features of a single light field image under an optical deep cascade structure, the present invention constructs a variety of optical feature combinations under the coupling of simple and complex optical modulations in a single optical layer, and uses a single optical layer to achieve feature expression capabilities that match the deep cascade structure of traditional optical neural networks. It is a new optical network architecture that perfectly solves the problem that traditional optical neural networks cannot be deeply cascaded due to energy loss, error accumulation and other reasons.

[0088] On the basis of the above technical solution, the training process of the neural network provided by the present invention is as follows:

[0089] The data set uses the public data set Qcuikdraw data set or Fashion-MNIST data set:

[0090] The Quickdraw dataset selects 10 categories (clock, chair, computer, glasses, tent, snowflake, pants, hurricane, flower, crown), with 250 images as training set, 50 images as validation set, and 50 images as test set (a total of 3500 images); each image format is a 40×40 grayscale image after optical transmission modulation, and the pixel value ranges from 0 to 255.

[0091] The Fashion-MNIST dataset includes 10 categories: T-shirts, pants, pullovers, dresses, coats, sandals, shirts, sneakers, bags, and ankle boots; each category takes 5500 images as training sets, 500 images as validation sets, and 1000 images as test sets (a total of 70000 images); each image format is a 28×28 grayscale image with pixel values ​​ranging from 0 to 255.

[0092] With the change of training methods, optical neural networks based on width learning theory are based on specific optical network structures, perform width parallelism in spatial dimensions, and use batch training with backpropagation to continuously optimize the modulation matrix to achieve higher accuracy.

[0093] Among them, the image X in the data set enters the above-mentioned optical network framework, and based on the set hyperparameters such as learning rate, batch size, etc., the loss optimization training of the optical neural network based on back propagation is carried out; finally, the network loss value converges, and the optical neural network training is completed.

[0094] Save the trained model; in practical applications, when a new optical image is obtained, input it into the trained network model to obtain the corresponding light output vector; in the optical image classification task, it is the specific image category classification result; in the optical image reconstruction task, it is a compressed vector with image information, which can be used for image reconstruction later.

[0095] The effect of the present invention can be further illustrated by the following simulation:

[0096] (1) Simulation conditions

[0097] This example training uses CUDA version 12.2, accelerated by two NVIDIA GeForce RTX 4090 GPUs, each with 20 GB of memory, Python version 3.9.18, and PyTorch version 2.1.0.

[0098] The simulation experiment is carried out in the above environment.

[0099] (2) Evaluation indicators

[0100] For simulation experiments, qualitative and quantitative analysis are used to evaluate the algorithm performance. The main evaluation indicators used in quantitative analysis are:

[0101] ① Classification task accuracy (Accuracy): the ratio of correct detections to all pixels;

[0102] ② Structural Similarity Index (SSIM) for Reconstruction Tasks: A method for evaluating the similarity between two images;

[0103]

[0104] Where x and y represent two images. x and μ y Respectively represent the brightness mean (i.e. average grayscale) of images x and y. x and σ y Respectively represent the brightness standard deviation of images x and y (i.e. the standard deviation of pixel values, used to measure contrast). xy Represents the brightness covariance of images x and y (i.e., measures the correlation between the brightness of the two images). C1 and C2 are two constants used to avoid the denominator being zero.

[0105] ③Reconstruction task peak signal-to-noise ratio (PSNR): a widely used image quality evaluation indicator used to measure the difference between the original image and the distorted image;

[0106]

[0107] Among them, (MAX I ) is the maximum possible pixel value in the image. MSE (Mean Squared Error) is the mean square error between the original image and the distorted image.

[0108] ④Reconstruction task mean square error (MSE): a commonly used error measurement method used to evaluate the difference between the predicted value and the actual value;

[0109]

[0110] Where I(i,j) is the pixel value of the original image (or reference image) at position (i,j). K(i,j) is the pixel value of the distorted image (or predicted image) at position (i,j). M and N are the height and width of the image, respectively.

[0111] (3) Simulation experiment

[0112] The present invention evaluates and verifies the performance of the proposed optical neural network based on width learning theory in image classification and image reconstruction tasks. These datasets include the Quickdraw dataset for classification and reconstruction and the Fashion-MNIST dataset for classification.

[0113] The Qcuikdraw dataset is easy to binarize and display on the optical neural network. The present invention selects 10 categories (clock, chair, computer, glasses, tent, snowflake, pants, hurricane, flower, crown). The training and validation datasets contain 300 images per category (3000 images in total), and each category is randomly divided into 250 and 50 images for training and validation, respectively. The test set contains 50 images per category (500 images in total), which are completely different from the images in the training and validation sets.

[0114] The Fashion-MNIST dataset is a direct replacement for the traditional MNIST dataset, sharing the same image dimensions but offering a higher complexity, making it suitable for validating the robustness of models. It includes 10 classes: T-shirts, Pants, Pullovers, Dresses, Coats, Sandals, Shirts, Sneakers, Bags, and Ankle Boots. The training set contains 6,000 images per class (60,000 images in total), while the test set contains 1,000 images per class (10,000 images in total). Each example is a 28×28 grayscale image with pixel values ​​ranging from 0 to 255.

[0115] Simulation 1:

[0116] Under the experimental condition of 16 feature nodes, the Tanhshrink function was selected as the nonlinearity, and a three-layer optical neural network classifier based on width learning theory was established. The classifier has a simple structure and an accuracy of 83.8% on the Quickdraw dataset and 88% on the Fashion-MNIST dataset, as shown in Table 1. The research results are significantly improved over the traditional three-layer optical classifier and the five-layer optical classifier. Importantly, due to the parallel distribution of the width learning optical neural network, it can more fully utilize the highly parallel characteristics of light transmission compared to the sequential distribution of traditional optical neural networks. Therefore, it shows enhanced noise resistance and enhanced robustness, highlighting its superior performance.

[0117] Table 1 Comparison of the accuracy of different network models on two datasets

[0118]

[0119] In addition, from Figure 5The convergence curves in Figure 2 show that the optical neural network based on width learning theory has a convergence speed advantage of one to two orders of magnitude compared to the optical fully connected network (MLP). Although the optical neural network based on width learning theory is a lightweight model with significantly fewer parameters than the deep cascade model, its performance is better than the three-layer cascade MLP architecture and comparable to the five-layer cascade MLP. Without increasing the experimental equipment, the parallel distribution of the optical neural network based on width learning theory can more effectively utilize the highly parallel characteristics of optical transmission than the serial distribution of traditional optical neural networks, thereby achieving faster convergence speed and stronger noise resistance.

[0120] Simulation 2:

[0121] In order to more realistically simulate the test conditions of optical networks under actual physical constraints, we further verified the performance advantages of parallel distribution of optical neural networks based on width learning theory over traditional sequential distribution, especially in the presence of error accumulation. During the linear modulation process, we introduced random errors in the modulation matrix of each layer. As the number of layers increases, the increase in errors shows a clear amplification trend, and the errors tend to double cumulatively as the number of layers increases. In addition to introducing random errors in the linear modulation process of each layer, the verification was carried out under optimal conditions.

[0122] Since the two datasets have different difficulties in classification, we added different levels of error to improve the clarity of the results. The results show that on the Quickdraw dataset shown in Table 2, the relative accuracy of the optical neural network based on width learning theory decreased by 1.03%, while the relative accuracy of the three-layer classifier and the five-layer classifier decreased by 30.29% and 37.48%, respectively.

[0123] Table 2 Noise accuracy of different network models on the Quickdraw dataset

[0124]

[0125] On the Fashoon-MNIST dataset as shown in Table 3, the relative accuracy of the optical neural network based on the width learning theory drops by 1.65%, while the relative accuracy of the three-layer classifier and the five-layer classifier drops by 16% and 68.01%, respectively. The comparative analysis based on the relative accuracy drop further emphasizes the robustness to noise.

[0126] Table 3 Noise accuracy of different network models on the Fashoon-MNIST dataset

[0127]

[0128] In summary, for both datasets, the accuracy of the optical neural network based on width learning theory is higher than that of the three-layer classifier, while the accuracy is slightly lower than that of the five-layer classifier, e.g. Figure 6 As shown. Adding random errors to the training model significantly changes the trend of the presented curves compared to the curves without errors. The accuracy decreases more with the increase of network size, indicating that errors accumulate with the increase of depth and eventually have a significant impact on the accuracy. Therefore, compared with the sequential distribution of optical networks, the parallel distribution of optical neural networks based on width learning theory significantly improves its resistance to misalignment errors, resulting in a smaller absolute accuracy decrease and a smaller relative accuracy decrease. This shows that optical neural networks based on width learning theory have better overall robustness.

[0129] Simulation three:

[0130] Image reconstruction tasks are another classic application of optical neural networks. Due to their fast modulation capabilities and low energy consumption, optical neural networks have found practical use in various scenarios such as remote sensing and automotive radar. In these applications, optical computing methods are used for preprocessing to compress the information. The preprocessed image is further processed by a back-end electronic neural network, reducing the energy consumption and complex complexity of the entire processing pipeline.

[0131] For this task, we use a previously trained optical neural network to train a new digital backend to perform image compression. We use different optical neural networks, including optical multilayer perceptrons (optical MLPs) and optical neural networks based on width learning theory, to compress the image to a simplified set of points. The compressed data is then passed to the same digital backend network for reconstruction. We evaluate and compare the compression performance of different optical neural networks by evaluating the structural similarity index (SSIM), peak signal-to-noise ratio (PSNR), and mean square error (MSE) between the reconstructed image and the original image.

[0132] In order to objectively compare the compression performance of different networks, we uniformly compress 40 × 40 Quickdraw images into 4 vectors, and the compression ratio obtained is 400: 1. The SSIM, PSNR, and MSE between the reconstructed images and the original images are calculated for a total of 500 reconstructed images and the average is taken.

[0133] Table 4 Reconstruction error of different network models on the Quickdraw dataset

[0134] Network Model MLP-3 Optical neural network based on width learning theory SSIM 0.5287 0.5364 PSNR 16.1128 16.6649 MSE 0.0307 0.0264

[0135] As shown in Table 4, these results show that the reconstruction performance of the optical neural network based on width learning theory is slightly better than that of the optical MLP model. These results show that, firstly, both the optical MLP and the optical neural network based on width learning theory generate low-dimensional feature spaces, which preserve the complex properties of the original image to a certain extent, which can be seen in Figure 7 This shows that optical neural networks can be used not only for classification tasks, but also for preprocessing tasks to reduce the complexity of different front-end targets. Secondly, under the same or fewer optical devices, optical neural networks based on width learning theory not only outperform traditional cascade optical neural networks in classification tasks, but also outperform traditional cascade optical neural networks, showing superior noise resistance.

[0136] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An optical neural network based on width learning theory, characterized in that: It includes a characteristic modulation module, an enhancement modulation module, a combination connection module and an optical classification module connected in sequence, wherein: The feature modulation module is used to perform feature modulation processing on the input light field image to generate parallel feature node images; during the feature modulation process, the light field image is copied into multiple light field image copies, and modulated with a spatial light modulation matrix of the same size as the light field image, so as to achieve the output of multiple parallel feature node images of the same size and different modulations; The enhanced modulation module is used to perform enhanced modulation processing on the input parallel feature node images to generate multiple enhanced node images that have undergone nonlinear optical modulation; before the enhanced modulation process, the parallel feature node images are first subjected to beam separation to obtain two sets of identical parallel feature node images; during the enhanced modulation process, one set of parallel feature node images enters the combined connection module, and the other set of parallel feature node images uses the optical nonlinear layer to perform nonlinear modulation of the image, thereby achieving the output of multiple enhanced node images of the same size; The combination connection module is used to combine the input parallel feature node image and the enhanced node image to generate a combined node image; during the combination connection process, the two beams of the parallel feature node image and the enhanced node image are distributed in parallel, thereby realizing the output of the combined node image; The optical classification module is used to perform optical classification on the input combined node image to generate a light output vector; during the optical classification process, the combined node image is processed by optical linear matrix multiplication to generate a light output vector.

2. The optical neural network based on width learning theory according to claim 1, characterized in that: The feature modulation module includes a digital micromirror device unit, a microlens array unit, and a spatial light modulator unit, wherein: The digital micromirror device unit uses an incoherent broadband illumination field to perform image loading processing on an input optical image to obtain an input data field; The microlens array unit adopts optical fan-out to generate multiple parallel optical image copies of the input data field to obtain a parallel input data field; The spatial light modulator unit adopts a spatial light modulation matrix to perform matrix weighted modulation on each optical image copy in the parallel input data field to obtain parallel characteristic node images.

3. The optical neural network based on width learning theory according to claim 1, characterized in that: The enhanced modulation module comprises a beam splitter unit and an optical nonlinear layer unit, wherein: The beam splitter unit adopts an optical beam splitter for splitting the light field transmission beam of the parallel characteristic node image into two identical beams, thereby obtaining two sets of identical parallel characteristic node images; The optical nonlinear layer unit specifically utilizes the nonlinear photoelectric effect to perform nonlinear image modulation processing on a set of parallel characteristic node images input to obtain enhanced node images.

4. The optical neural network based on width learning theory according to claim 1, characterized in that: The optical nonlinear layer does not change the shape or size of the light beam, but changes the intensity of the light to varying degrees, thereby achieving nonlinear adjustment.

5. The optical neural network based on width learning theory according to claim 1, characterized in that: The combined connection module comprises a relay lens unit, wherein: The relay lens unit adopts relay lenses with different focal lengths, does not change the size of the image or the information content it carries, and performs imaging at different positions to achieve optical shaping; the two beams of the parallel feature node image and the enhanced node image are spliced ​​together in parallel at the same place using the relay lens unit to obtain a combined node image.

6. The optical neural network based on width learning theory according to claim 1, characterized in that: The optical classification module comprises a microlens array unit, a spatial light modulator unit, and a lens unit, wherein: The microlens array unit adopts optical fan-out to generate a corresponding input light field for the combined node image; The spatial light modulator unit uses a spatial light modulator of matrix multiplication to linearly modulate the input light field to obtain a light field output image equal to the number of task categories; The lens unit adopts an optical lens to converge the light field output image to obtain the final light output vector.

7. The optical neural network based on width learning theory according to claim 1, characterized in that: The training of the optical neural network based on width learning theory adopts a back propagation algorithm; After the optical image to be processed is obtained, it is input into the trained network model to obtain the corresponding light output vector.

8. The optical neural network based on width learning theory according to claim 1, characterized in that: The optical output vector is used for image classification or image reconstruction.

9. An optical image processing unit, characterized in that: The optical image processing unit is provided with a trained optical neural network model based on width learning theory according to any one of claims 1 to 8, which is used to classify or reconstruct the input optical image.

10. A computer-readable storage medium, wherein a computer program is stored in the medium; characterized in that: When the computer program is executed by a processor, the classification or reconstruction process of the optical image by the optical neural network based on the width learning theory according to any one of claims 1 to 8 is implemented.

Images