Computing Device and System Based on Deep Optical Neural Network Model

Abstract

The present disclosure relates to the field of optical computing technologies, and particularly to a deep optical neural network model, architecture, and system. Among them, the model includes: a set of optical computing modules, wherein the connection manner between the optical computing modules in the set of optical computing modules is determined by the target computing task, and the optical computing module includes an optical computing matrix, and the number of rows and columns of the optical computing matrix are determined by the target computing task. The present disclosure adopting the above solution can implement a complex intelligent model with a high number of parameters through the optical computing module.

Description

Technical Field

[0001] The present disclosure relates to the field of optical computing technologies, and particularly to a deep optical neural network model, architecture, and system. Background Art

[0002] With the rapid development of the fields of artificial intelligence and scientific computing, the complexity and scale of computing requirements are also increasing continuously. However, existing electronic computing technologies are limited by Moore's Law, and their performance is gradually approaching the saturation state, making it difficult to effectively meet the increasingly stringent requirements for computing power and power consumption of large-scale complex algorithms. Light has natural advantages such as high throughput and low latency during propagation. Optical computing technology that uses photons instead of electrons as the computing carrier is regarded as the key to breaking the existing computing bottleneck. Summary of the Invention

[0003] The present disclosure aims to at least partly solve one of the technical problems in the related art.

[0004] To this end, the first object of the present disclosure is to propose a deep optical neural network model to implement a complex intelligent model with a large number of parameters through an optical computing module.

[0005] The second object of the present disclosure is to propose a deep optical neural network architecture.

[0006] The third object of the present disclosure is to propose a deep optical neural network system.

[0007] To achieve the above object, the first aspect embodiment of the present disclosure proposes a deep optical neural network model, including:

[0008] A set of optical computing modules, wherein the connection manner between the optical computing modules in the set of optical computing modules is determined by a target computing task, and the optical computing module includes an optical computing matrix, and the number of rows and columns of the optical computing matrix are determined by the target computing task.

[0009] Optionally, the target computing task is an optical residual neural network computing task, the set of optical computing modules includes multiple subsets of optical computing modules, the multiple subsets of optical computing modules are connected in series, the optical computing modules in the subset of optical computing modules are connected in series, the number of rows and columns of the optical computing matrix of the optical computing modules in the same subset of optical computing modules are the same, the optical computing matrix is used to implement at least one convolution kernel, and the input end and the output end of the subset of optical computing modules are connected across layers inside the subset of optical computing modules.

[0010] Optionally, the model further includes an electronic module. The two subsets of optical computing modules connected in series are respectively a first subset of optical computing modules and a second subset of optical computing modules. The first subset of optical computing modules and the second subset of optical computing modules are connected in series through the electronic module. Among them,

[0011] The input ends of the electronic module are respectively connected to the input end and the output end of the first subset of optical computing modules. The output ends of the electronic module are respectively connected to the input end and the output end of the second subset of optical computing modules. The electronic module is configured to perform non-linear calculations on the optical input signal and the optical output signal of the first subset of optical computing modules, and use the non-linear calculation result as an optical input signal to input into the second subset of optical computing modules.

[0012] Optionally, the electronic module is further configured to:

[0013] Temporarily store the non-linear calculation result.

[0014] Optionally, the target calculation task is an optical Transformer calculation task. The model includes multiple Transformer decoding modules connected in series. The set of optical computing modules includes multiple subsets of optical computing modules. Each Transformer decoding module includes one subset of optical computing modules. The optical computing modules in the subset of optical computing modules are connected in series and / or in parallel to splice the optical computing matrix into a high-dimensional optical computing matrix corresponding to the optical Transformer calculation task.

[0015] Optionally, the Transformer decoding module includes a self-attention sub-module and a feed-forward natural network sub-module. The self-attention sub-module includes the subset of optical computing modules. The input ends of the self-attention sub-module are respectively connected to the output end of the self-attention sub-module, the input end of the feed-forward natural network sub-module, and the output end of the feed-forward natural network sub-module.

[0016] Optionally, the self-attention sub-module further includes a first optical linear layer and a second optical linear layer. Among them,

[0017] The input ends of the first optical linear layer are respectively connected to the output end of the second optical linear layer, the input end of the feed-forward natural network sub-module, and the output end of the feed-forward natural network sub-module. The output end of the first optical linear layer is connected to the input end of the subset of optical computing modules. The output end of the subset of optical computing modules is connected to the input end of the second optical linear layer.

[0018] Optionally, at least one third optical linear layer is included in the feedforward natural network sub-module.

[0019] To achieve the above object, an embodiment of the second aspect of the present disclosure provides a deep optical neural network architecture, including: at least one deep optical neural network model shown in any one of the foregoing first aspects.

[0020] To achieve the above object, an embodiment of the third aspect of the present disclosure provides a deep optical neural network system, including: at least one deep optical neural network model shown in any one of the foregoing first aspects.

[0021] In summary, the deep optical neural network model, architecture, and system provided by the present disclosure can evolve matrix operations of different dimensions and scales through reasonable reconstruction and structural combination of the optical computing module according to the target computing task, and perform vertical depth expansion. The optical computing module chips can be stacked into a neural network computing structure that can execute the target computing task, and the depth and performance of the network can be effectively expanded. Compared with general optical neural networks, the deep optical neural network model can effectively stack the overall number of parameters (i.e., effectively improve performance as the number of parameters increases), and can implement complex intelligent models with a large number of parameters.

[0022] Additional aspects and advantages of the present disclosure will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and / or additional aspects and advantages of the present disclosure will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, where:

[0024] Figure 1 is a schematic structural diagram of a deep optical neural network model provided by an embodiment of the present disclosure;

[0025] Figure 2 is a schematic architecture diagram of an optical ResNet model provided by an embodiment of the present disclosure;

[0026] Figure 3 is an application schematic diagram of an optical ResNet model provided by an embodiment of the present disclosure;

[0027] Figure 4 is a schematic architecture diagram of an optical Transformer model provided by an embodiment of the present disclosure;

[0028] Figure 5 is an application schematic diagram of an optical Transformer model provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] Embodiments of the present disclosure will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where like or similar reference numerals denote like or similar elements or elements having like or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present disclosure, but should not be construed as limiting the present disclosure.

[0030] With the rapid development of Artificial Neural Network (ANN) technology, the performance and complexity of machine vision algorithms have been significantly improved, leading to a continuous increase in the demand for high computing power. The bottleneck of traditional electronic computing architectures lies in their high power consumption and limited computing speed, which cannot fully meet the high parallel processing capabilities required for dynamic machine vision processing.

[0031] However, there are inherent difficulties in implementing complex intelligent tasks with light. To handle complex intelligent tasks, it is often necessary to expand the depth of the neural network. Extremely deep neural networks have stronger expressive power and can learn more complex non-linear mappings. This enables them to perform well in processing high-dimensional and complex data, such as tasks like image recognition and natural language processing. However, for most analog computing architectures, the lack of reconstruction and expansion capabilities restricts the stacking of their depth and also limits their ability to handle intelligent tasks. Existing optical computing models are often limited to simple fully connected or convolutional operations when implementing neural networks, and the relatively fixed structure of each computing unit also makes it difficult to combine these modules into large-scale and intelligent effective deep optical neural networks. Therefore, even with a large number of structures stacked, it is difficult for them to exhibit the intelligence of deep neural networks. This results in most optical neural network work remaining at the processing of simple data sets and the application in idealized scenarios.

[0032] The present disclosure will be described in detail below in conjunction with specific embodiments.

[0033] Figure 1 The following is a schematic structural diagram of a deep optical neural network model provided for an embodiment of the present disclosure. As Figure 1 shown, the deep optical neural network model includes:

[0034] An optical computing module set, wherein the connection mode between the optical computing modules in the optical computing module set is determined by the target computing task, and the optical computing module includes an optical computing matrix, and the number of rows and columns of the optical computing matrix is determined by the target computing task.

[0035] According to some embodiments, the number of rows and columns of the optical computing matrix respectively correspond one-to-one to the input signal dimension and the output signal dimension in the optical computing module. For example, the number of rows of the optical computing matrix is the same as the input signal dimension in the optical computing module, and the number of columns of the optical computing matrix is the same as the output signal dimension in the optical computing module; or, the number of rows of the optical computing matrix is the same as the output signal dimension in the optical computing module, and the number of columns of the optical computing matrix is the same as the input signal dimension in the optical computing module.

[0036] In some embodiments, the optical computing module can be reconstructed and expanded, the input signal dimension and the output signal dimension can be adjusted according to the target computing task, and the number of rows and columns of the optical computing matrix can also be adjusted according to the target computing task.

[0037] According to some embodiments, the target computing task refers to a task that requires a deep optical neural network model for calculation. The target computing task includes but is not limited to tasks such as image classification and natural language processing.

[0038] It should be noted that due to the flexible reconstruction characteristics of the optical computing module, the present disclosure can successfully implement the special structures of each layer of the model in the design of the deep optical neural network, rather than simply splicing fixed-form computing units. This is also reflected in the scaling law of the model. As the scale of network parameters increases, the performance of the deep optical neural network model gradually improves when dealing with complex image classification and language modeling problems. For example, in the language modeling task, when the number of network parameters increases from 5M, 30M, 60M, 0.117B to 0.345B parameters, the loss function decreases from 4.91 to 3.04, and the accuracy rate increases from 28.58% to 43.92%.

[0039] It is easy to understand that by reasonably reconstructing and combining the structures of the optical computing module according to the target computing task, evolving matrix operations of different dimensions and scales, and performing vertical deep expansion, the optical computing module chips can be stacked into a neural network computing structure that can execute the target computing task, and can effectively expand the depth and performance of the network. Compared with general optical neural networks, this deep optical neural network model can effectively stack the overall number of parameters (that is, effectively improve the performance as the number of parameters increases), and can implement complex intelligent models with a large number of parameters.

[0040] Optionally, the target computing task is an optical Residual Neural Network (ResNet) computing task. The set of optical computing modules includes multiple subsets of optical computing modules, which are connected in series. The optical computing modules within each subset of optical computing modules are connected in series. The number of rows and columns of the optical computing matrix of the optical computing modules in the same subset of optical computing modules is the same. The optical computing matrix is used to implement at least one convolutional kernel. The input end and the output end of the subset of optical computing modules are cross-connected within the subset of optical computing modules.

[0041] According to some embodiments, the ResNet computing task refers to a computing task that needs to be executed using ResNet.

[0042] In some embodiments, the number of rows and columns of the optical computing matrix of the optical computing modules in different subsets of optical computing modules may be the same or different, and can be specifically determined by grouping the set of optical computing modules according to the target computing task.

[0043] Exemplarily, Figure 2 is a schematic diagram of the architecture of an optical ResNet model provided by an embodiment of the present disclosure; wherein, the boxes represent optical computing modules, 64 / 64, 128 / 128, 256 / 256, 512 / 512 represent the number of rows and columns of the optical computing matrix, and 3×3, 3×3 / 2 represent the dimensions of the convolutional kernels. As Figure 2 shown, by deeply connecting multiple optical computing modules with different input and output scales in series, multiple 3×3 convolutional kernels can be implemented inside each optical computing module. Then, these optical computing modules are grouped, and cross-layer connections from input directly to output are made within the subset of optical computing modules. Therefore, each optical computing module contains the grouping and summation of the calculation results of multiple convolutional kernels. By performing combined calculations on the outputs of the optical computing modules and then passing them into a similar structure in the next layer, convolutional layers and deep networks can be implemented, thereby enabling an optical ResNet.

[0044] According to some embodiments, the optical ResNet model further includes an electronic module. The two subsets of optical computing modules connected in series are respectively a first subset of optical computing modules and a second subset of optical computing modules. The first subset of optical computing modules and the second subset of optical computing modules are connected in series through the electronic module; wherein,

[0045] The input end of the electronic module is respectively connected to the input end and the output end of the first subset of optical computing modules. The output end of the electronic module is respectively connected to the input end and the output end of the second subset of optical computing modules. The electronic module is used to perform non-linear calculations on the optical input signal and the optical output signal of the first subset of optical computing modules, and use the non-linear calculation result as the optical input signal to input into the second subset of optical computing modules.

[0046] In some embodiments, by performing optoelectronic nonlinear operations between subsets of optical computing modules using an electronic module, the representation ability of the optical ResNet model can be improved.

[0047] In some embodiments, the electronic module can also be used to temporarily store the results of the nonlinear calculation. Therefore, when the electronic module performs nonlinear calculations on the optical input signal and the optical output signal of the first subset of optical computing modules, it can directly receive the optical output signal temporarily stored in the previous electronic module.

[0048] In some embodiments, when performing nonlinear calculations, the electronic module can read out the optical output signal through optoelectronic conversion (ADC) and then perform combined calculations.

[0049] In some embodiments, the electronic module can, for example, adopt a Field Programmable Gate Array (FPGA).

[0050] Taking a scenario as an example, Figure 3 is an application schematic diagram of an optical ResNet model provided by an embodiment of the present disclosure. As Figure 3 shown, the optical ResNet model is formed by stacking more than a hundred layers of optical computing modules. In an image classification task, the structure and scale of ResNet can be utilized to accurately identify and classify images in various complex scenarios. Whether it is common objects in daily life or specific images in professional fields, accurate classification results can be obtained, which can reflect the intelligence of the deep optical network.

[0051] Optionally, the target computing task is an optical Transformer computing task, the model includes multiple serially connected Transformer decoding modules, the set of optical computing modules includes multiple subsets of optical computing modules, and each Transformer decoding module includes a subset of optical computing modules. The optical computing modules in the subset of optical computing modules are serially connected and / or parallely connected to splice the optical computing matrix into a high-dimensional optical computing matrix corresponding to the optical Transformer computing task.

[0052] According to some embodiments, the Transformer computing task refers to a computing task that needs to be performed using a Transformer.

[0053] According to some embodiments, the Transformer decoding module includes a self-attention sub-module and a feed-forward neural network sub-module. The self-attention sub-module includes a subset of optical computing modules. The input ends of the self-attention sub-module are respectively connected to the output end of the self-attention sub-module, the input end of the feed-forward neural network sub-module, and the output end of the feed-forward neural network sub-module.

[0054] In some embodiments, the self-attention sub-module further includes a first optical linear layer and a second optical linear layer; wherein,

[0055] The input ends of the first optical linear layer are respectively connected to the output end of the second optical linear layer, the input end of the feed-forward neural network sub-module, and the output end of the feed-forward neural network sub-module. The output end of the first optical linear layer is connected to the input end of the subset of optical computing modules, and the output end of the subset of optical computing modules is connected to the input end of the second optical linear layer.

[0056] In some embodiments, the feed-forward neural network sub-module includes at least one third optical linear layer.

[0057] In some embodiments, the first optical linear layer, the second optical linear layer, and the third optical linear layer can be implemented by an electrical computing module. The electrical computing module can be, for example, an electronic module similar to an FPGA.

[0058] It should be noted that by performing optoelectronic non-linear operations between subsets of optical computing modules using the first optical linear layer, the second optical linear layer, and the third optical linear layer, the representation ability of the optical Transformer model can be improved.

[0059] Exemplarily, Figure 4 is a schematic diagram of the architecture of an optical Transformer model provided by an embodiment of the present disclosure. As Figure 4 shown, by reconstructing each optical computing module and then connecting them in parallel and in series in a certain number, small matrices are stitched together into a high-dimensional optical computing matrix (for example, a 128×128 matrix is split into 128 128×1 matrices, each small matrix can be implemented by an optical computing module, and the outputs of the optical computing modules are arranged together to form a 128-dimensional vector). Then, an optical linear layer for implementing non-linearity and hierarchical connection is introduced at the output end, and the optical Transformer is realized by connecting them in series layer by layer. This process can not only improve the representation ability of the optical Transformer model but also effectively utilize the advantages of optical computing to accelerate complex network operations.

[0060] Taking a scenario as an example, Figure 5 is an application schematic diagram of an optical Transformer model provided by an embodiment of the present disclosure. As Figure 5As shown, applying the present disclosure to the fields of language processing and text-to-image (generating images from text) can successfully construct an optical intelligent model based on transformers with a parameter count in the billions. In the text-to-image task, this optical Transformer model can generate images that highly match the input text description, providing new possibilities for creative design and content creation.

[0061] In summary, the model provided in this embodiment can implement complex neural network structures such as optical ResNet and optical Transformer through a unique combination method based on reconfigurable optical computing modules, and can successfully solve the problems that the existing optical networks have small depth and scale, and it is difficult for the performance to effectively improve as the number of parameters increases. This innovation not only promotes the development of the field of optical computing, but also provides new ideas and directions for cutting-edge technologies such as artificial intelligence and deep learning. It has broad application prospects and is expected to bring new opportunities for high-performance intelligent computing in the post-Moore era and real-time analysis and control of transient scientific phenomena, and can be applied to fields such as unmanned systems, autonomous driving, and ultrafast science.

[0062] To implement the above embodiment, the present disclosure also proposes a deep optical neural network architecture, including: at least one deep optical neural network model provided in the foregoing embodiment.

[0063] To implement the above embodiment, the present disclosure also proposes a deep optical neural network system, including: at least one deep optical neural network model provided in the foregoing embodiment.

[0064] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the present disclosure all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0065] It should be noted that personal information from users should be collected for legal and reasonable purposes and should not be shared or sold outside of these legal uses. In addition, such collection / sharing should be carried out after obtaining the informed consent of the user, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization including authorizing relevant user information before the user uses the function. In addition, any necessary steps should be taken to safeguard and protect access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.

[0066] The present disclosure is expected to provide an implementation scheme that allows users to selectively prevent the use or access of personal information data. That is, the present disclosure is expected to provide hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, the risk can be minimized by restricting data collection and deleting the data. In addition, when applicable, personal identifiers are removed from such personal information to protect the privacy of users.

[0067] In the technical solution of the present disclosure, the acquisition, transmission, storage, use, processing, etc. of data all comply with the relevant provisions of national laws and regulations.

[0068] It should be noted that in the embodiments of the present disclosure, some existing solutions in the industry such as certain software, components, models, etc. may be mentioned. They should be regarded as exemplary, and their purpose is only to illustrate the feasibility in the implementation of the technical solution of this application, but it does not mean that the applicant has already or necessarily used this solution.

[0069] In the description of the foregoing embodiments, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0070] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0071] Any process or method description shown in a flowchart or described in other ways herein can be understood as representing a module, segment, or part of code including one or more executable instructions for implementing a customized logic function or process. The scope of the preferred embodiments of the present disclosure includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in a reverse order according to the involved functions, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0072] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definable list of executable instructions for implementing logical functions, and can be embodied specifically in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or used in conjunction with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection portion having one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which a program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other suitable processing as necessary, and then stored in a computer memory.

[0073] It should be understood that various parts of the present disclosure can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having suitable combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0074] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the method of the above embodiments can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0075] In addition, each functional unit in various embodiments of the present disclosure may be integrated into one processing module, may exist physically alone for each unit, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0076] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A computing device based on a deep optical neural network model, characterized in that, Applied to image recognition or natural language processing tasks, the deep optical neural network model includes: An optical computing module set, wherein the connection mode between the optical computing modules in the optical computing module set is determined by the target computing task, the optical computing module includes an optical computing matrix, and the number of rows and columns of the optical computing matrix are determined by the target computing task; Wherein, the target computing task is an optical residual neural network computing task, the optical computing module set includes multiple optical computing module subsets, the multiple optical computing module subsets are connected in series, the optical computing modules in the optical computing module subset are connected in series, the number of rows and columns of the optical computing matrix of the optical computing modules in the same optical computing module subset are the same, the optical computing matrix is used to implement at least one convolution kernel, and the input end and the output end of the optical computing module subset are connected across layers inside the optical computing module subset.

2. The device according to claim 1, characterized in that The model further includes an electronic module. The two optically connected computing module subsets connected in series are respectively a first optically connected computing module subset and a second optically connected computing module subset. The first optically connected computing module subset and the second optically connected computing module subset are connected in series through the electronic module; wherein, The input end of the electronic module is respectively connected to the input end and the output end of the first optically connected computing module subset, and the output end of the electronic module is respectively connected to the input end and the output end of the second optically connected computing module subset. The electronic module is used to perform non-linear calculations on the optical input signal and the optical output signal of the first optically connected computing module subset, and use the non-linear calculation result as the optical input signal to input to the second optically connected computing module subset.

3. The device according to claim 2, characterized in that The electronic module is further used to: Temporarily store the non-linear calculation result.

4. A computing device based on a deep optical neural network model, characterized in that, Applied to image recognition or natural language processing tasks, the deep optical neural network model includes: An optical computing module set, wherein the connection mode between the optical computing modules in the optical computing module set is determined by the target computing task, the optical computing module includes an optical computing matrix, and the number of rows and columns of the optical computing matrix are determined by the target computing task; Wherein, the target computing task is an optical Transformer computing task, the model includes multiple serially connected Transformer decoding modules, the optical computing module set includes multiple optical computing module subsets, and each Transformer decoding module includes one of the optical computing module subsets. The optical computing modules in the optical computing module subset are connected in series and / or in parallel to splice the optical computing matrix into a high-dimensional optical computing matrix corresponding to the optical Transformer computing task.

5. The device according to claim 4, characterized in that The Transformer decoding module includes a self-attention sub-module and a feed-forward natural network sub-module. The self-attention sub-module includes the subset of optical computing modules. The input end of the self-attention sub-module is respectively connected to the output end of the self-attention sub-module, the input end of the feed-forward natural network sub-module, and the output end of the feed-forward natural network sub-module.

6. The device according to claim 5, characterized in that The self-attention sub-module further includes a first optical linear layer and a second optical linear layer; wherein, The input end of the first optical linear layer is respectively connected to the output end of the second optical linear layer, the input end of the feed-forward natural network sub-module, and the output end of the feed-forward natural network sub-module. The output end of the first optical linear layer is connected to the input end of the subset of optical computing modules, and the output end of the subset of optical computing modules is connected to the input end of the second optical linear layer.

7. The device according to claim 5, characterized in that, The feed-forward natural network sub-module includes at least one third optical linear layer.

8. A deep optical neural network system, characterized in that, Comprising: The device according to any one of claims 1 to 3, or the device according to any one of claims 4 to 7.

Images