Intelligent light computing lifelong learning architecture system and apparatus

Abstract

The application discloses a smart optical computing lifelong learning architecture system and device, the system comprises: a multispectral characterization layer, which is used for converting original input containing multiple tasks into coherent light of different wavelengths through multispectral characterization of the electric signal; a lifelong learning optical neural network layer, which comprises a cascaded sparse light convolution layer in the Fourier plane of an optical system, and the different wavelengths of the coherent light of the input cascaded sparse light convolution layer are trained step by step through the lifelong learning optical neural network, and the final spatial light signal is output through the lifelong learning optical neural network layer; and an electrical network readout layer, which is used for identifying the final optical output data detected by the final spatial light signal to obtain a multiple task recognition result. The application realizes multiple task and high-performance machine intelligent computing, learns each task by adaptively activating sparse light connection in the coherent light field, and gradually obtains experience information for various tasks by gradually expanding the activated connection.

Description

Technical Field

[0001] This invention relates to the field of machine learning task technology, and in particular to an intelligent optical computing lifelong learning architecture system and device. Background Technology

[0002] Driven by massive datasets, machine learning tasks are becoming increasingly diverse and complex. One unresolved issue in machine intelligence is how AI agents can propagate in a more intelligent way and possess powerful learning capabilities to progressively learn multiple tasks. With the end of Moore's Law, energy consumption has become a major obstacle to the widespread adoption of current electrical neural network (ANN) methods for broader tasks, especially in edge devices. There is an urgent need to find next-generation computing paradigms to overcome the physical limitations of ANNs. Large-scale intelligent computing is the primary guarantee for realizing increasingly rich and complex machine learning tasks. Currently, AI based on traditional electrical computing processors faces power consumption limitations, hindering their sustainable performance improvements.

[0003] Light is a computing paradigm that overcomes the inherent limitations of electrical computing, improving energy efficiency, processing speed, and computational throughput by orders of magnitude. These remarkable properties have been used to build application-specific optical architectures to solve fundamental mathematical and signal processing problems, achieving performance far exceeding that of existing electronic processors. Simple visual processing tasks, such as handwritten digit recognition and saliency detection, have been effectively validated using wave-optical simulations or small-scale optical computing systems. Meanwhile, some works combine optical computing units with various electrical neural networks to expand the scale and flexibility of ONNs, such as depth optics, Fourier neural networks, and hybrid optoelectronic convolutional networks. However, traditional optical-based implementations are limited to small-scale applications and cannot continuously learn from experience across multiple tasks to adapt to new environments. The main reason is that they inherit a common problem from traditional electrical computing systems: learning new knowledge disrupts previously learned knowledge, and previously learned tasks are rapidly forgotten when training on new tasks—a phenomenon known as "catastrophic forgetting." These existing ONNs fail to fully utilize the inherent sparsity and parallelism of light, ultimately resulting in poor network capacity and scalability for large-scale machine learning tasks.

[0004] Unlike humans, who possess the ability to gradually absorb, learn, and memorize knowledge, optical neural networks (ONNs) are unique in that they only function when a task requires processing. Two crucial neurocognitive mechanisms are involved: sparse neuronal connections and parallel task processing, which together facilitate lifelong learning in the human brain. Therefore, in ONNs, the inherent sparsity and parallelism of optical operators can be naturally extended from biological neurons to optical neurons. This optical computing framework, mimicking the structure and function of the human brain, demonstrates its potential to alleviate the aforementioned problems and shows more advantages than electrical neural networks in constructing feasible lifelong learning computing systems. Summary of the Invention

[0005] The present invention aims to at least partially solve one of the technical problems in the related art.

[0006] To this end, this invention proposes an intelligent optical computing lifelong learning architecture system and designs a lifelong learning optical neural network (L... 2 ONN (Optical Network Interface) is used to achieve multi-tasking and high-performance machine intelligence. Benefiting from the inherent sparsity and parallelism in massive optical connections, L... 2 ONN naturally mimics the lifelong learning mechanism of neurons and synapses in the human brain. It learns each task by adaptively activating sparse optical connections in a coherent light field, while gradually acquiring experiential information about various tasks by progressively expanding the activated connections. The multi-task optical features are processed in parallel by multispectral characterizations assigned different wavelengths.

[0007] Another objective of this invention is to propose an intelligent optical computing lifelong learning architecture device.

[0008] To achieve the above objectives, this invention proposes an intelligent optical computing lifelong learning architecture system, which includes a multispectral representation layer, a lifelong learning optical neural network layer, and an electrical network readout layer, wherein...

[0009] The multispectral characterization layer is used to characterize the original input electrical signal containing multiple tasks into coherent light of different wavelengths through multispectral representation.

[0010] The lifelong learning optical neural network layer includes cascaded sparse light convolutional layers in the Fourier plane of the optical system. The lifelong learning optical neural network is trained stepwise by multi-tasks on coherent light of different wavelengths input to the cascaded sparse light convolutional layers, and the final spatial light signal is output through the lifelong learning optical neural network layer.

[0011] The electrical network readout layer is used to identify the final optical output data obtained from the final spatial light signal detection in order to obtain the multi-task recognition result.

[0012] In addition, the intelligent optical computing lifelong learning architecture system according to the above embodiments of the present invention may also have the following additional technical features:

[0013] Furthermore, in one embodiment of the present invention, each sparse light convolutional layer includes an optical modulation filter and an optical diffraction unit. The optical system converts coherent light of different wavelengths into sparse light features and inputs them into the cascaded sparse light convolutional layers for optical convolution operations. The optical modulation filter adaptively activates the sparse light features after optical convolution operations using optical neurons and inputs the activated optical neurons into the optical diffraction unit to modulate the optical neuron connections of each single task to output the final spatial light signal.

[0014] Furthermore, in one embodiment of the present invention, the electrical network readout layer is also used to detect the final spatial light signal on the output plane using an intensity sensor to obtain the final optical output data.

[0015] Furthermore, in one embodiment of the present invention, the optical modulation filter is an optical modulation filter based on phase change material PCM, wherein the PCM includes GST units; each GST unit contains two states, amorphous and crystalline, corresponding to different spectral transmittances; at the same wavelength, the GST units with spectral transmittance higher than a preset threshold are in an activated state, and the GST units with spectral transmittance lower than the preset threshold are in an inactive state.

[0016] Furthermore, in one embodiment of the present invention, the optical system is a 4f optical system; preset multi-task optical features. It is the i-th task in the k-th sparse optical convolutional layer in the spectrum λ i The feature representation on the above is transformed into a Fourier transform using the first 2f system:

[0017]

[0018] in, This represents the optical eigenmap in the Fourier domain, where F represents the Fourier transform matrix, and is modulated using an optical modulation filter. for:

[0019]

[0020] in, M represents the characteristics of the modulated light. k Represents the phase modulation matrix, I k (λ i ) represents the intensity modulation matrix; using a second 2f system to The inverse Fourier transform is applied to the spatial domain, and the regularized optical output data is detected on the output plane using an intensity sensor.

[0021]

[0022] Excluding the electrical network readout layer, the optical output data of each sparse optical convolutional layer is... Remapped to the input of the next layer:

[0023]

[0024] Here, remap() represents the corresponding nonlinear operation in optical computation.

[0025] Furthermore, in one embodiment of the present invention, the electrical network readout layer is also used to transmit the final spatial light output data detected on the output plane based on the intensity sensor. The light intensity data of the light blocks is cut into l preset sizes, and the light intensity data of the light blocks is input to the electrically fully connected layer to output the multi-task recognition result; where n is the number of optical module layers.

[0026] Furthermore, in one embodiment of the present invention, the lifelong learning optical neural network layer is also used for:

[0027] For training each task on the optical modulation filter, a dense activation map is trained using a lifelong learning optical neural network. i And use the intensity threshold (thres) to map i Pruning to a sparse activation graph:

[0028] map i [map i <thres]=0

[0029] Among them, map i Represents the activation map on the i-th task; optical neurons with light intensity data above the intensity threshold are retained and activated:

[0030]

[0031] Where ΔW represents the backpropagation gradient matrix of the optical convolution weight W, the ∧ operation represents finding the intersecting unit of two matrices, and the V operation gradually merges each activation map matrix.

[0032] The loss function of the lifelong learning optical neural network is:

[0033]

[0034] Where L CEN P represents the softmax cross-entropy loss. i and G i α represents the network prediction and the true value of the data for the i-th task, respectively, and α represents the regularization coefficient.

[0035] Furthermore, in one embodiment of the invention, the optical modulation filter is also used to share optical weights learned from all tasks.

[0036] Furthermore, in one embodiment of the present invention, the optical modulation filter based on phase change material (PCM) is all-optically switched, and the optical modulation filter based on PCM is also used to adaptively activate optical neurons in the spatial and spectral dimensions of the input light field.

[0037] To achieve the above objectives, another aspect of the present invention proposes an intelligent optical computing lifelong learning architecture device, including a multispectral characterization module, a beam splitter, a mirror, a lens, an optical modulation filter, and an intensity sensor;

[0038] An electrical signal containing multiple tasks is input to the multispectral characterization module to be characterized as coherent light of different wavelengths through multispectral characterization. The coherent light of different wavelengths is guided and modulated by a beam splitter, a reflector, a lens, and an optical modulation filter to output a final spatial light signal. The intensity sensor detects the final spatial light signal to obtain the final optical output data, and obtains the multi-task recognition result of the final optical output data through the output plane.

[0039] The intelligent optical computing lifelong learning architecture system and device of this invention realize multi-task and high-performance machine intelligent computing, avoid the catastrophic forgetting problem of ordinary optical neural networks (ONN), and complete multi-task lifelong learning on a number of challenging tasks (visual classification, speech recognition, medical diagnosis, etc.).

[0040] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0041] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0042] Figure 1 This is a schematic diagram of the structure of an intelligent optical computing lifelong learning architecture system according to an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram illustrating the learning principle of an optical lifelong learning network according to an embodiment of the present invention;

[0044] Figure 3 This is an architecture diagram of the L2ONN optical lifelong learning network according to an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of optical lifetime learning of L2ONN on a representative visual classification task according to an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram illustrating the numerical performance evaluation of L2ONN according to an embodiment of the present invention;

[0047] Figure 6 This is a schematic diagram of the intelligent optical computing lifelong learning architecture device according to an embodiment of the present invention. Detailed Implementation

[0048] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0049] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0050] The intelligent optical computing lifelong learning architecture system and apparatus proposed according to embodiments of the present invention are described below with reference to the accompanying drawings.

[0051] Figure 1 This is a schematic diagram of the intelligent optical computing lifelong learning architecture system according to an embodiment of the present invention.

[0052] like Figure 1 As shown, the system 10 includes a multispectral characterization layer 100, a lifelong learning optical neural network layer 200, and an electrical network readout layer 300, wherein...

[0053] Multispectral characterization layer 100 is used to characterize the original input electrical signal containing multiple tasks into coherent light of different wavelengths through multispectral representation;

[0054] The lifelong learning optical neural network layer 200 includes cascaded sparse light convolutional layers in the Fourier plane of the optical system. The lifelong learning optical neural network is trained stepwise by multi-tasks on coherent light of different wavelengths input to the cascaded sparse light convolutional layers, and the final spatial light signal is output through the lifelong learning optical neural network layer 200.

[0055] The electrical network readout layer 300 is used to identify the final optical output data obtained from the final spatial light signal detection in order to obtain the multi-task recognition result.

[0056] It is understandable that the L proposed in this invention 2 The principle of ONN optical lifelong learning is as follows: Figure 2 As shown, inspired by brain neural morphology, L 2 ONN learns multiple tasks progressively within a single model with extremely efficient computation. This invention is the first to unlock the unique characteristics of optical sparsity and multispectral representation in an optical computing architecture, endowing ONN with lifelong learning capabilities similar to the human brain.

[0057] In one embodiment of the present invention, such as Figure 2 The diagram illustrates the optical lifelong learning principle of this invention. Figure 2The term 'a' in this context refers to lifelong learning in humans. It explains that the brain can gradually absorb, learn, and remember knowledge throughout its lifespan. Neurons and synapses function only when activated by corresponding signals, with active neurons being relatively sparse, and information is transmitted through parallel tasks. Humans possess an extraordinary ability to retain memories and gradually absorb new knowledge throughout their lives. The brain can progressively absorb, learn, and remember knowledge, for example, developing from recognizing basic characters and objects to understanding complex scenarios. During learning, neurons and synapses are gradually activated and connected to remember a specific task, functioning only when there are task-related external stimuli. In the human brain, lifelong learning is achieved through sparse neuronal connections and parallel task processing.

[0058] Figure 2 In the diagram, 'b' represents the optical lifelong learning graph proposed in this invention. The optical computing module continuously improves its learning ability and memorizes knowledge. Optical neurons are continuously learned and activated during incremental learning. Input information from different tasks is encoded into coherent light of different wavelengths, which are then processed by the sparse optical convolution module to obtain the final inference result. Furthermore, at each stage of incremental learning, a new set of optical neurons is activated. These updated neurons encode newly learned knowledge and will be consolidated to avoid catastrophic forgetting in future learning, just as humans never forget basic skills they have learned, such as how to ride a bicycle.

[0059] Figure 2 In this context, 'c' represents the proposed L. 2 A schematic diagram of ONN. The input to the incremental learning task is encoded in coherent light fields with different wavelengths, and simultaneously fed in parallel to cascaded sparse optical convolutional modules. Optical features are further processed and inference results are computed through light wave propagation and sparse neuron activation. With progressive incremental learning, L... 2 ONN gained a wealth of experience and knowledge by adapting to a number of challenging tasks in new scenarios.

[0060] In one embodiment of the present invention, such as Figure 3 As shown, this invention employs a sparse optical modulation filter based on phase change materials to modulate the optical neuron connections for each individual task; simultaneously, it constructs a multispectral optical convolution module based on optical diffraction to extract multi-task features assigned to different wavelengths. Throughout the architecture, optical neurons are selectively activated based on their input connections. Unlike existing ONNs that attempt to mimic the ANN architecture, L... 2 ONN's optical lifelong learning is designed based on the inherent physical properties of light propagation, thus fully realizing the potential of optical computing.

[0061] For example, Figure 3 In the diagram, 'a' represents the architecture of the sparse optical convolution module. The inputs from multiple tasks are projected onto a module with multispectral representation. In the coherent optical field, the original input is an electrical signal, and the signal projected into the optical field is a multispectral optical signal. Beam splitters (BS), mirrors (M), lenses (L), and optical modulation filters are used to guide and modulate light propagation. The cascaded sparse optical convolutional layers are implemented by arranging optical modulation filters on the Fourier plane of the 4f optical system. With the optical output O detected on the output plane, the final result can be obtained through the electrical network readout layer. Figure 3 In the diagram, 'b' represents the detailed structure of the sparse optical convolutional layer. Each layer receives sparse features as input. The modulation filter based on phase change material (PCM) is all-optically switched, which performs adaptive activation of optical neurons in the spatial and spectral dimensions on the input light field, and then sends the activated optical neurons to the subsequent optical diffraction module. Figure 3 In the diagram, 'c' represents the training strategy for incremental optical learning on an 8×8 optical modulation filter. Training for each task initially involves learning a dense activation map. i The activation maps are further pruned into sparse maps using an intensity threshold (thres). The final optical neuron activation maps learned for each task are preserved and remain unchanged in subsequent learning evolutions, and the optical modulation filters share the optical weights learned from all tasks.

[0062] Specifically, Figure 3 In L, 'a' represents L. 2 The overall architecture of ONN and the principle of this invention are as follows: First, the input is converted into a multispectral representation carrying multi-task information, projected onto a shared domain, i.e., onto a spatial light representation, and propagated through an optical computing module based on optical diffraction. The optical computing module consists of cascaded sparse optical convolutional layers in the Fourier plane of a coherent 4f optical system. Each layer contains an adaptively switching optical modulation filter based on different tasks, and an optical diffraction unit that can selectively activate optical neurons based on the input data. The final spatial light output of the sparse optical convolutional module is detected on the plane by an intensity sensor and further fed into an electrical network readout layer to obtain the recognition result. The implementation method may include the following steps:

[0063] Assumption It is the i-th task in the k-th optical convolutional layer in the spectrum λ i The feature representation of a 2f system is first used to transform its Fourier transform to:

[0064]

[0065] in Let F represent the optical eigenmap in the Fourier domain, and let F represent the Fourier transformation matrix. Then, The light-modulated filter further modulates the light into:

[0066]

[0067] in M represents the characteristics of the modulated light. k Represents the phase modulation matrix, I k (λ i The ) represents the intensity modulation matrix, which can dynamically activate or prune optical neuron connections to link different tasks. Next, another 2f system is used to... The inverse Fourier transform back to the spatial domain, and its regularized output It will be detected by the intensity sensor on the output plane:

[0068]

[0069] Except for the last electrical network readout layer, the output of each layer will be remapped to the input of the next layer:

[0070]

[0071] Where `remap()` represents the corresponding nonlinear operation in optical computation, defined as the final spatial light output of the sparse optical convolution module with n optical module layers (set to 3 in the experiment). The light intensity will be detected by an intensity sensor on a plane, and the plane will be cut into 14×14 small spatial blocks. The light intensity of each spatial block will be collected and sent to a 196×10 electrically fully connected layer to obtain the final recognition result.

[0072] Furthermore, Figure 3 The detailed structure of a single sparse optical convolutional layer is analyzed in section b. Each layer receives sparse optical features from the previous layer and performs optical convolution. Exemplarily, a phase change material (PCM) is used as an optical modulation filter to switch the activation of spatial and spectral dimensions, which are fed back into the optical diffraction module to modulate the optical neuron connections. The proposed PCM consists of GeSbTe (GST) grown on a transparent silicon substrate. Each GST unit has both amorphous and crystalline states with different spectral transmittances, which can be switched instantly by a conversion light. All-optical control ensures that phase and intensity modulation occurs without delay. At the same wavelength, the present invention defines GST units with higher transmittance as the activated state and units with lower transmittance as the inactivated state.

[0073] Furthermore, Figure 3 c in the figure shows L using an 8×8 optical modulation filter. 2 The ONN training strategy aims to achieve the desired lifetime learning of optical signals through training. All PCM units remain inactive initially and gradually activate during training. For each new task, the optical modulation filter first learns a dense activation map, which is then further pruned into a sparse activation map using an intensity threshold (thres).

[0074] map i [map i <thres] = 0,

[0075] map i This represents the activation map on the i-th task. Only neurons with light intensities above the intensity threshold are retained and remain active in subsequent tasks.

[0076]

[0077] Where ΔW represents the backpropagation gradient matrix of the optical convolution weights W, the ∧ operation represents finding the intersecting units of the two matrices, and the V operation gradually merges each activation map matrix. The optical modulation filter shares optical weights learned from all known tasks and gradually acquires multi-task experience to adapt to new environments, avoiding the catastrophic forgetting problem. During training, the loss function is set as follows:

[0078]

[0079] Where L CEN P represents the softmax cross-entropy loss. i and G i α represents the network prediction and the true value of the data for the i-th task, respectively, and α represents the regularization coefficient.

[0080] Furthermore, Figure 4 For L 2 ONN's lifelong learning of light on representative visual classification tasks. Figure 4 In the example, 'a' represents five basic MNIST datasets used for incremental optical learning. Figure 4 b, L 2 ONN and Figure 4 c in the diagram represents the activation map of the first layer of optical neurons in the original ONN. Through network learning, L... 2 The optical neuron connections in ONN are initially sparse and continuously activated, colored red, yellow, green, blue, and purple respectively, whereas in the original ONN, they are very dense from the very first task. Figure 4 d, L 2 Comparison of training curves between ONN and the original ONN. Training for each task consists of 5 iterations, L... 2 ONNs can continuously increase their abilities and learn all the tasks they see, while ordinary ONNs quickly forget what they have learned and fall into a catastrophic forgetfulness zone of less than 20%.

[0081] This invention validated the three-layer L-type array with a size of 200×200 on five representative visual classification tasks. 2ONN's lifelong learning ability ( Figure 4 ) and its numerical performance ( Figure 5 ). Figure 4 In the figure, 'a' shows five benchmark datasets similar to MNIST. This invention progressively trains L on these five tasks. 2 ONN, and in Figure 4 The evolution of the activation map of optical neurons in layer 1 (b) is obtained, which gradually expands and remains fixed during task training. For each task training, this invention observes L... 2 ONNs only require the activation of a small subset of optical neurons to learn their experiential knowledge.

[0082] For comparison, this invention constructs a 200×200 three-layer original ONN and a computationally equivalent five-layer electrical network LeNet and learns incrementally in the same way. Figure 4 c in the figure shows the changes in the activation map of optical neurons in the first layer of the original ONN, which remains dense throughout the training process. The activation of optical neurons for new tasks tends to completely occupy the space and interfere with previously learned neurons, leading to a catastrophic forgetting problem. Figure 4 d in L is compared 2 The training convergence curves between the ONN and the original ONN were obtained by applying 25 iterations, with 5 iterations per task. Setting 20% ​​as the catastrophic forgetting baseline, it was observed that the original ONN exhibited catastrophic forgetting after only 2 iterations of training for a new task, indicating that previously learned experience was almost entirely erased. Through network training, L... 2 ONNs continuously learn all the tasks they see and acquire abilities for new tasks, while ordinary ONNs quickly forget what they've learned and fall into a catastrophic forgetting state. Using an activation threshold of 0.5, L... 2 ONN can incrementally learn up to 14 tasks, utilizing 96.3% of optical neuron activations, while achieving an energy efficiency ratio more than an order of magnitude higher than LeNet-5, which is based on electrical networks.

[0083] Furthermore, Figure 5 For L 2 Numerical performance evaluation of ONN. Figure 5 In the 'a', the original ONN, L 2 Accuracy comparison between ONN and LeNet based on different benchmarks. The pruning rate (70%) used in the LeNet electrical network is compared with that of L... 2 ONN has a minimum sparsity close to that of ONN and uses the same training strategy to learn multiple tasks step by step. Figure 5 In step b, the relationship between network sparsity and performance is evaluated using a single FashionMNIST task. All networks are configured with a fixed pruning rate at the same sparsity. Figure 5In line c, the evolution of the activation map of the optical modulation filter under various training sequences is shown. The five tasks are categorized into three difficulty levels based on the optical neuron activation maps (row 1) required to train a single task. Tasks 1 and 2, as well as tasks 3 and 4, have the same difficulty level because they occupy similar activation densities. Based on this criterion, Figure 5 In line 'd', evaluate the impact of training order from easy to difficult and difficult to easy on network performance (lines 2 and 3). Figure 5 The 'e' in the text further reports the impact of changing the order of tasks within the same difficulty level on network performance (lines 4 and 5).

[0084] Furthermore, Figure 5 The report in section a presents the original ONN based on training a single task and L based on incremental optical learning. 2 Accuracy comparison between ONN and different benchmarks of electrical incremental learning-based electrical network ANN (LeNet). 2 The calculation of the equivalent size of ONN applies the same principles as L. 2 The ONN achieves a similar pruning rate (70%) to the minimum sparsity and is trained using the same training strategy. During the learning process, the L-type ONN with highly sparse optical convolutions outperforms the fully densely connected original ONN. 2 ONN loses at most 1.9% of its accuracy, yet uses only 34.3% of the parameters of the original ONN to acquire empirical knowledge for all five tasks. Regarding the comparison of incremental learning capabilities, it is comparable to L... 2 Compared to ONN, the electrical neural network LeNet achieves 1.2% higher accuracy on the first task, but lower accuracy on all other tasks. More importantly, due to the lack of inherent sparsity, the performance of the electrical neural network drops rapidly during training on the fourth task.

[0085] Furthermore, Figure 5 b evaluated the original ONN and L 2 A performance comparison of ONN and LeNet on the FashionMNIST task with different sparsity levels. This invention shows that when the sparsity is less than 40%, LeNet outperforms the ONN-based method; however, its performance significantly decreases if the sparsity exceeds 60%. When the sparsity reaches 99%, LeNet... 2 The ONN robustly achieved 82.6% accuracy (a decrease of only 3.1%), while the original ONN achieved 53.8%, and the electrical network LeNet achieved 22.3%. This invention concludes that optical devices, due to their large amount of optical information, have more inherent advantages over electronic devices in terms of sparsity and parallelism, enabling them to achieve equivalent or higher performance with fewer computational resources. This naturally demonstrates the potential to mimic the efficient biological mechanisms of lifelong learning in humans.

[0086] Furthermore, Figure 5 The study in section c investigated how the learning order affects L 2 The performance of ONN optical lifelong learning. First, this invention trains L on each individual task. 2 ONNs were analyzed, and the activation density of optical neurons in their first layer was determined, which was used as an evaluation criterion for task difficulty levels. Therefore, the five tasks were divided into three difficulty levels, with tasks 1 and 2, and tasks 3 and 4 having similar densities. Under this criterion, L... 2 ONN is trained using two extreme training sequences: from easy to difficult and from difficult to easy. The corresponding accuracy curves are compared as follows: Figure 4 As shown in d. This invention observes that, compared to training from hard to hard, training from easy to hard consumes fewer optical neuron activations in all steps (up to 23.25%), but achieves higher performance on all tasks (up to 10.42%). L 2 ONN demonstrates its human-like characteristics in lifelong learning, requiring a gradual process to absorb, memorize, and consolidate skills. Starting with complex tasks has the opposite effect, much like a person learns to crawl before walking. Furthermore, this invention sequentially moves the internal order of difficulty levels 1 and 2, and... Figure 5 The evaluation results are reported in the e-report. Although the spatial shape of the activation of optical neurons showed differences, the obtained density and accuracy remained almost unchanged with the basic training order (from easy to difficult). 2 ONN demonstrated its high learning capacity, versatility, and extremely high energy efficiency, providing a key solution for achieving more advanced AI tasks.

[0087] In summary, this invention learns each task by adaptively activating sparse optical neuron connections using a PCM-based optical modulation filter, while gradually acquiring experiential knowledge for various tasks by progressively expanding the optical activation map. Multi-task optical features are processed in parallel by multispectral representations assigned different wavelengths. Except for the linear activation and electrical network readout layers, all computations are performed using optical devices. The principle of optical lifelong learning is inspired by the brain's memory preservation mechanisms and the adaptation to new knowledge through the utilization of sparse neuron connections and parallelized task processing. Due to its inherent massive optical information, optical computing has more inherent advantages over electrical computing systems in terms of sparsity and parallelism, naturally mimicking the biological mechanisms of human lifelong learning. Unlike existing artificial intelligence methods that disrupt previously learned experiential knowledge when training for new tasks, the proposed optical lifelong learning architecture possesses the ability to continuously master multiple tasks, avoiding the problem of catastrophic forgetting. In conclusion, this invention has demonstrated the effectiveness of the proposed L... 2 ONN provides a key solution for large-scale, real-world AI applications, offering unprecedented scalability and versatility. 2ONN has demonstrated exceptional learning capabilities in challenging machine learning tasks such as visual classification, speech recognition, and medical diagnosis, supporting a variety of novel environments. This invention anticipates that the proposed method will accelerate the development of more powerful optical computing as a key support for modern advanced machine intelligence, and usher in a new era of artificial intelligence.

[0088] The intelligent optical computing lifelong learning architecture system according to embodiments of the present invention is used to realize multi-tasking and high-performance machine intelligence. Benefiting from the inherent sparsity and parallelism in large-scale optical connections, L... 2 ONN naturally mimics the lifelong learning mechanism of neurons and synapses in the human brain. It learns each task by adaptively activating sparse optical connections in a coherent light field, while gradually acquiring experiential information about various tasks by progressively expanding the activated connections. Multi-task optical features are represented by multispectral characteristics allocated with different wavelengths and processed in parallel. This invention endows machine intelligence with the ability to compute at the speed of light, while simultaneously making optical computing unprecedentedly scalable and versatile.

[0089] To achieve the above embodiments, such as Figure 6 As shown, this embodiment also provides an intelligent optical computing lifelong learning architecture device 1, which includes a multispectral characterization unit 2, a beam splitter 3, a reflector 4, a lens 5, an optical modulation filter 6, an optical diffraction unit 7, and an intensity sensor 8.

[0090] The electrical signal containing multiple tasks is input to the multispectral characterization unit 1 to be characterized as coherent light of different wavelengths through multispectral characterization. The coherent light of different wavelengths is guided and modulated by the beam splitter 3, the mirror 4, the lens 5, the optical modulation filter 6 and the optical diffraction unit 7 to output the final spatial light signal. The intensity sensor 8 detects the final spatial light signal to obtain the final optical output data, and obtains the multi-task recognition result of the final optical output data through the output plane.

[0091] The present invention discloses an intelligent optical computing lifelong learning architecture device for realizing multi-tasking and high-performance machine intelligence. Benefiting from the inherent sparsity and parallelism in large-scale optical connections, it learns each task by adaptively activating sparse optical connections in a coherent optical field, while gradually acquiring experiential information for various tasks by progressively expanding the activated connections. Multi-task optical characteristics are characterized by multispectral representations allocated with different wavelengths and processed in parallel. This invention endows machine intelligence with the ability to compute at the speed of light, while simultaneously enabling optical computing to possess unprecedented scalability and versatility.

[0092] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0093] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A smart optical computing lifelong learning architecture system, characterized in that, The system includes a multispectral characterization layer, a lifelong learning optical neural network layer, and an electrical network readout layer, wherein... The multispectral characterization layer is used to characterize the original input electrical signal containing multiple tasks into coherent light of different wavelengths through multispectral representation. The lifelong learning optical neural network layer includes cascaded sparse optical convolutional layers in the Fourier plane of the optical system. It performs multi-task progressive training of the lifelong learning optical neural network on coherent light of different wavelengths input to the cascaded sparse optical convolutional layers, and outputs the final spatial light signal through the lifelong learning optical neural network layer. The optical system is... Optical system; preset multi-task optical features It is the first The first sparse optical convolutional layer One task in the spectrum Feature representation on, using the first 2 The system performs a Fourier transform: in, Represents the light feature mapping in the Fourier domain. Representing the Fourier transform matrix, modulated using an optical modulation filter. for: in, Indicates the characteristics of the modulated light. Represents the phase modulation matrix. Represents the intensity modulation matrix; using the second 2 The system will The inverse Fourier transform is applied to the spatial domain, and the regularized optical output data is detected on the output plane using an intensity sensor. : . Excluding the electrical network readout layer, the optical output data of each sparse optical convolutional layer is... Remapped to the input of the next layer: in, This represents the corresponding nonlinear operation in optical computing; The electrical network readout layer is used to identify the final optical output data obtained from the final spatial light signal detection in order to obtain the multi-task recognition result.

2. The intelligent optical computing lifelong learning architecture system according to claim 1, characterized in that, Each sparse light convolutional layer includes an optical modulation filter and an optical diffraction unit. The optical system converts coherent light of different wavelengths into sparse light features and inputs them into the cascaded sparse light convolutional layers for optical convolution operations. The optical modulation filter adaptively activates the sparse light features after optical convolution operations using optical neurons and inputs the activated optical neurons into the optical diffraction unit to modulate the optical neuron connections of each individual task to output the final spatial light signal.

3. The intelligent optical computing lifelong learning architecture system according to claim 1, characterized in that, The electrical network readout layer is also used to detect the final spatial light signal on the output plane using an intensity sensor to obtain the final optical output data.

4. The intelligent optical computing lifelong learning architecture system according to claim 2, characterized in that, The optical modulation filter is an optical modulation filter based on phase change material PCM. The PCM includes GST units. Each GST unit contains two states: amorphous and crystalline, corresponding to different spectral transmittances. At the same wavelength, GST units with spectral transmittance higher than a preset threshold are in an activated state, and GST units with spectral transmittance lower than the preset threshold are in an inactive state.

5. The intelligent optical computing lifelong learning architecture system according to claim 4, characterized in that, The electrical network readout layer is also used to process the final spatial light output data detected on the output plane based on the intensity sensor. Cut into A spatial block of a preset size is selected, and the light intensity data of the spatial block is input to an electrically fully connected layer to output the multi-task recognition result; wherein... This represents the number of optical module layers.

6. The intelligent optical computing lifelong learning architecture system according to claim 5, characterized in that, The lifelong learning optical neural network layer is also used for: For training each task on the optical modulation filter, a dense activation map is trained using a lifelong learning optical neural network. and using intensity threshold Will Pruning to a sparse activation graph: in, Representative at the Activation maps for each task; retaining activation of optical neurons whose light intensity data is above the intensity threshold: in, Represents optical convolution weights The backpropagation gradient matrix, The operation represents finding the cells where two matrices intersect. The operation then gradually merges each activation graph matrix; The loss function of the lifelong learning optical neural network is: in This represents the softmax cross-entropy loss. and These represent the network prediction and the ground truth value for the i-th task, respectively. This represents the regularization coefficient.

7. The intelligent optical computing lifelong learning architecture system according to claim 6, characterized in that, The optical modulation filter is also used to share optical weights learned from all tasks.

8. The intelligent optical computing lifelong learning architecture system according to claim 4, characterized in that, The optical modulation filter based on phase change material (PCM) is all-optically switched, and it is also used to perform adaptive optical neuron activation in the spatial and spectral dimensions of the input light field.

9. A smart optical computing lifelong learning architecture device, characterized in that, The device includes a multispectral characterization layer, a lifelong learning optical neural network layer, and an electrical network readout layer, wherein... The multispectral characterization layer is used to characterize the original input electrical signal containing multiple tasks into coherent light of different wavelengths through multispectral representation. The lifelong learning optical neural network layer includes cascaded sparse optical convolutional layers in the Fourier plane of the optical system. It performs multi-task progressive training of the lifelong learning optical neural network on coherent light of different wavelengths input to the cascaded sparse optical convolutional layers, and outputs the final spatial light signal through the lifelong learning optical neural network layer. The optical system is... Optical system; preset multi-task optical features It is the first The first sparse optical convolutional layer One task in the spectrum Feature representation on, using the first 2 The system performs a Fourier transform: in, Represents the light feature mapping in the Fourier domain. Representing the Fourier transform matrix, modulated using an optical modulation filter. for: in, Indicates the characteristics of the modulated light. Represents the phase modulation matrix. Represents the intensity modulation matrix; using the second 2 The system will The inverse Fourier transform is applied to the spatial domain, and the regularized optical output data is detected on the output plane using an intensity sensor. : . Excluding the electrical network readout layer, the optical output data of each sparse optical convolutional layer is... Remapped to the input of the next layer: in, This represents the corresponding nonlinear operation in optical computing; The electrical network readout layer is used to identify the final optical output data obtained from the final spatial light signal detection in order to obtain the multi-task recognition result.

Images