Large-scale reconfigurable photon large matrix model and architecture

Abstract

The invention relates to the technical field of optical computing, in particular to a large-scale reconfigurable photon large matrix model and architecture. Wherein the model comprises a mixed light calculation sub-model; the large-scale reconfigurable photon large matrix model comprises an acquisition module, a parallel input module and an integration module, and the acquisition module is used for acquiring a first large matrix and an input vector which need to be calculated; the parallel input module is used for inputting the first large matrix and the input vector into the mixed light calculation sub-model of each parallel channel in parallel based on the determined number of the parallel channels to obtain an output vector of each parallel channel; and the integration module is used for integrating the output vectors of the parallel channels to obtain a target output vector. According to the scheme, under the condition that high calculation precision and high data flux are considered, a large-scale general optical matrix operation new normal form is achieved, and therefore a complex optical general calculation task is supported.

Description

Technical Field

[0001] The present disclosure relates to the field of optical computing technology, and in particular to a large-scale reconfigurable photon matrix model and architecture. Background Art

[0002] With the rapid development of artificial intelligence and scientific computing, the complexity and scale of computing needs are also increasing. However, the performance of existing large-scale matrix computing is gradually approaching saturation, making it difficult to effectively cope with the increasingly stringent demands of large-scale complex algorithms for computing power and power consumption. Light has natural advantages such as high throughput and low latency during propagation. Optical computing technology that uses photons instead of electrons as computing carriers is seen as the key to breaking the existing computing bottleneck. Summary of the invention

[0003] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0004] To this end, the first objective of the present disclosure is to propose a large-scale reconfigurable photon matrix model to achieve high-performance optical computing.

[0005] The second objective of the present disclosure is to propose a large-scale reconfigurable photonic matrix architecture.

[0006] To achieve the above-mentioned purpose, the first embodiment of the present disclosure proposes a large-scale reconfigurable photon matrix model, including: a hybrid optical computing sub-model; the large-scale reconfigurable photon matrix model includes an acquisition module, a parallel input module and an integration module, wherein The acquisition module is used to acquire the first largest matrix and input vector to be calculated; The parallel input module is used to input the first large matrix and the input vector in parallel into the mixed light calculation sub-model of each parallel channel based on the determined number of parallel channels, so as to obtain the output vector of each parallel channel; The integration module is used to integrate the output vectors of each parallel channel to obtain a target output vector.

[0007] Optionally, the hybrid optical computing sub-model includes a diffraction coding compression module, an interference computing module and a diffraction decoding characterization module, wherein the output of the diffraction coding compression module is the input of the interference computing module, and the output of the interference computing module is the input of the diffraction decoding characterization module.

[0008] Optionally, the diffraction coding compression module is used to compress the first large matrix and the input vector into a second large matrix and a first vector; The interference calculation module is used to perform arbitrary matrix multiplication on the second largest matrix and the first vector to obtain a second vector; The diffraction decoding characterization module is used to calculate the second weight of diffraction decoding according to the second vector to obtain the output vector of each parallel channel.

[0009] Optionally, the interference calculation module is specifically used to: perform arbitrary matrix multiplication according to the second largest matrix and the first vector using an MZI or crossbar structure to obtain a second vector.

[0010] Optionally, the diffraction coding compression module is specifically used to: downsample the input vector to a first vector, and compress the first large matrix into a second large matrix through a first weight calculated by diffraction modulation.

[0011] Optionally, the integration module is specifically used to: perform weighted summation on the output vectors of each parallel channel to obtain a target output vector.

[0012] To achieve the above-mentioned purpose, the second aspect of the present disclosure proposes a large-scale reconfigurable photonic matrix architecture, including: at least one large-scale reconfigurable photonic matrix model shown in any one of the aforementioned first aspects.

[0013] Optionally, the architecture includes: a plurality of the hybrid optical computing sub-models.

[0014] Optionally, the plurality of mixed light computing sub-models are connected in parallel.

[0015] Optionally, the hybrid optical computing sub-model includes a diffraction coding compression module, an interference computing module and a diffraction decoding characterization module, wherein the output of the diffraction coding compression module is the input of the interference computing module, and the output of the interference computing module is the input of the diffraction decoding characterization module.

[0016] In summary, the large-scale reconfigurable photonic large matrix model and architecture provided by the present invention, through a multi-channel parallel strategy, parallelizes the first large matrix element array into multiple groups of finite-scale matrix element array operation channels, and realizes the parameter scale adaptive and precise fitting of arbitrary matrix multiplication operations through multiple hybrid optical computing sub-models. While taking into account high computing accuracy and high data throughput, a new paradigm of large-scale general optical matrix operations is realized, which can support complex optical general computing tasks.

[0017] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which: Figure 1 A schematic diagram of the structure of a large-scale reconfigurable photon matrix model provided by an embodiment of the present disclosure; Figure 2 A schematic diagram of the structure of a hybrid optical computing sub-model provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0020] Currently, for most analog computing architectures, signals may attenuate during transmission and processing, and noise may also affect the quality of the signal. This may lead to errors or instability in the calculation results. For optical computing models, due to factors such as material defects, wavefront errors, and uneven transmission in the optical system, various errors will inevitably be introduced in optical calculations. These errors may affect the accuracy of the calculation results and cause deviations between experimental results and simulation data. If the optical neural network parameters are expanded by simply stacking the number of layers, the errors will gradually accumulate during propagation, eventually causing huge errors in the output.

[0021] In the existing technology, the classification of four vowel sound wave signals can be completed through an array composed of 56 cascaded Mach-Zehnder Interferometers (MZI), or a deep diffraction neural network based on a cascade of optical diffraction masks can be used to achieve visual tasks such as handwritten digit recognition and image saliency detection; the splitting and aggregation of the optical path can give the diffracted light calculation the ability of multi-channel parallel processing.

[0022] However, the computing effect in the above-mentioned prior art is described by the classification / regression accuracy of a certain intelligent computing task, and cannot be measured by the precision deviation of a certain operation; and the above-mentioned optical computing is difficult to deeply cascade the computing operators, which limits the representation ability and computing scale of optical computing.

[0023] The present disclosure is described in detail below with reference to specific embodiments.

[0024] Figure 1 This is a schematic diagram of the structure of a large-scale reconfigurable photon matrix model provided by an embodiment of the present disclosure. Figure 1 As shown, the large-scale reconfigurable photon matrix model includes: a hybrid optical computing sub-model; wherein the large-scale reconfigurable photon matrix model includes an acquisition module, a parallel input module and an integration module, wherein: An acquisition module is used to obtain the first largest matrix and input vector that need to be calculated; A parallel input module, used for inputting the first largest matrix and the input vector in parallel into the mixed light calculation sub-model of each parallel channel based on the determined number of parallel channels, so as to obtain an output vector of each parallel channel; The integration module is used to integrate the output vectors of each parallel channel to obtain a target output vector.

[0025] In one embodiment of the present disclosure, the above-mentioned large-scale reconfigurable photonic large matrix model is suitable for large-matrix computing. Based on this, the above-mentioned large-scale reconfigurable photonic large matrix model can be applied to a variety of scenarios, such as the classification of intelligent computing tasks, or intelligent question-and-answer scenarios.

[0026] Among them, in one embodiment of the present disclosure, the large-scale reconfigurable photon matrix model can split the super-large parameter matrix encoding into multiple groups of parallel finite-scale parameter reconfigurable sub-matrices, and further decode and fuse the operation results of the sub-matrices into the final large matrix operation result output. And, in one embodiment of the present disclosure, the approximate fitting of arbitrary matrix multiplication operations can be achieved through the "diffraction-interference-diffraction" hybrid optical computing sub-model.

[0027] Specifically, in one embodiment of the present disclosure, the number of parallel channels can be determined according to the calculation accuracy requirement. The more parallel channels there are, the higher the corresponding calculation accuracy. After the number of parallel channels is determined based on the calculation accuracy requirement, each parallel channel corresponds to a hybrid light calculation sub-model, thereby constructing a hybrid light calculation model array, thereby achieving parameter scale adaptive and precise fitting of arbitrary matrix multiplication operations.

[0028] In one embodiment of the present disclosure, multiple parallel mixed light computing sub-models in the mixed light computing model array are completed through joint training, and each mixed light computing sub-model corresponds to its own weight parameter.

[0029] Specifically, in one embodiment of the present disclosure, Figure 2 Schematic diagram of the structure of a hybrid optical computing sub-model provided by an embodiment of the present disclosure. Figure 2 As shown, the above-mentioned hybrid optical computing sub-model includes a diffraction coding compression module, an interference computing module and a diffraction decoding characterization module, wherein the output of the diffraction coding compression module is the input of the interference computing module, and the output of the interference computing module is the input of the diffraction decoding characterization module.

[0030] In one embodiment of the present disclosure, the above-mentioned diffraction encoding and compression module is used to compress the first large matrix and the input vector into a second large matrix and a first vector; the interference calculation module is used to perform arbitrary matrix multiplication according to the second large matrix and the first vector to obtain a second vector; the diffraction decoding and characterization module is used to calculate the second weight of diffraction decoding according to the second vector to obtain the output vector of each parallel channel.

[0031] In one embodiment of the present disclosure, the above-mentioned diffraction encoding and compression module is specifically used to: downsample the input vector to a first vector, and compress the first large matrix into a second large matrix through the first weight of diffraction modulation calculation. Wherein, the dimension of the first vector is smaller than that of the input vector.

[0032] Exemplarily, in one embodiment of the present disclosure, assuming that the dimension of the input vector is N and the dimension of the first large matrix is N×N, the diffraction encoding and compression module can downsample the input vector to a first vector with dimension M (M < N), and compress the first large matrix with N channels into a second large matrix with M channels through the first weight of diffraction modulation calculation.

[0033] Moreover, in one embodiment of the present disclosure, the above-mentioned interference calculation module is specifically used to: perform arbitrary matrix multiplication according to the second large matrix and the first vector by using an MZI or crossbar structure to obtain a second vector.

[0034] Exemplarily, in one embodiment of the present disclosure, the interference calculation module can perform arbitrary matrix multiplication of M×M according to the second large matrix and the first vector by using an MZI or crossbar structure to obtain a second vector. Wherein, the dimension of the second vector is 1×M.

[0035] Further, in one embodiment of the present disclosure, when the above-mentioned diffraction decoding and characterization module calculates the second weight of diffraction decoding according to the second vector to obtain the output vector of each parallel channel, it can calculate the second weight of diffraction decoding on the M-dimensional second vector output by the interference calculation module and restore it to an N-dimensional output vector, that is, the dimension of the output vector is the same as that of the input vector.

[0036] Moreover, in one embodiment of the present disclosure, 100k groups of arbitrary input vectors can be selected, and 100k groups of true value operation outputs are obtained through operations on the matrix to be fitted with dimension N×N, thereby obtaining the dataset required for training the above-mentioned hybrid optical computing model array, and using this dataset to train the hybrid optical computing model array to obtain the weight parameters in each hybrid optical computing sub-model, so that the effect of optical computing approaches the operation effect of the actual matrix to be fitted.

[0037] Furthermore, in one embodiment of the present disclosure, the above-mentioned integration module is specifically used to: perform weighted summation on the output vectors of each parallel channel to obtain a target output vector.

[0038] In one embodiment of the present disclosure, the dimension of the large-scale matrix to be calculated is N*N, and the joint optimization calculation of multiple M*M parallel small matrices can be used to obtain the required results within the allowable error range. Among them, in the current Transformer-based large matrix operations, the matrix to be calculated generally has a strong sparsity (that is, the parameter redundancy is large). The use of the above-mentioned large-scale reconfigurable photon large matrix model can effectively reduce the number of multiplication and addition calculations in matrix operations, making the operation more efficient. At the same time, the above-mentioned matrix operation splitting method can theoretically enable small-scale computing hardware to support the calculation of larger matrices through reuse, and has strong scalability.

[0039] In summary, the model provided in this embodiment parallelizes the first large matrix element array into multiple groups of finite-scale matrix element array operation channels through a multi-channel parallel strategy, and realizes the parameter scale adaptive and precise fitting of arbitrary matrix multiplication operations through multiple hybrid optical computing sub-models. While taking into account high computing accuracy and high data throughput, a new paradigm of large-scale general optical matrix operations is realized, which can support complex optical general computing tasks, has broad application prospects, and can be applied to unmanned systems, autonomous driving, ultrafast science and other fields.

[0040] In order to implement the above embodiments, the present disclosure also proposes a large-scale reconfigurable photon matrix architecture, including: a large-scale reconfigurable photon matrix model provided by at least one of the above embodiments.

[0041] Optionally, the architecture includes: a plurality of hybrid optical computing sub-models, wherein the plurality of hybrid optical computing sub-models are connected in parallel.

[0042] Optionally, the hybrid optical computing sub-model includes a diffraction coding compression module, an interference computing module and a diffraction decoding characterization module, wherein the output of the diffraction coding compression module is the input of the interference computing module, and the output of the interference computing module is the input of the diffraction decoding characterization module.

[0043] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this disclosure shall comply with the relevant laws and regulations and shall not violate public order and good morals.

[0044] It should be noted that personal information from users should be collected for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign the agreement / authorization including authorization of relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others who have access to personal information data comply with its privacy policy and procedures.

[0045] The present disclosure anticipates providing implementation schemes for users to selectively block the use or access of personal information data. That is, the present disclosure anticipates providing hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, risks can be minimized by limiting data collection and deleting the data. In addition, when applicable, such personal information is de-identified to protect the privacy of the user.

[0046] The acquisition, transmission, storage, use, and processing of data in the technical solution disclosed in this disclosure are in compliance with the relevant provisions of national laws and regulations.

[0047] It should be noted that in the embodiments of the present disclosure, certain software, components, models and other existing solutions in the industry may be mentioned, which should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0048] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they contradict each other.

[0049] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0050] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.

[0051] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0052] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0053] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0054] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0055] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present disclosure. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present disclosure.

Claims

1. A large-scale reconfigurable photon matrix model, characterized in that: include: Hybrid optical computing sub-model; The large-scale reconfigurable photon matrix model includes an acquisition module, a parallel input module and an integration module, wherein: The acquisition module is used to acquire the first largest matrix and input vector to be calculated; The parallel input module is used to input the first large matrix and the input vector in parallel into the mixed light calculation sub-model of each parallel channel based on the determined number of parallel channels, so as to obtain the output vector of each parallel channel; The integration module is used to integrate the output vectors of each parallel channel to obtain a target output vector.

2. The large-scale reconfigurable photon matrix model according to claim 1 is characterized in that: The hybrid optical computing sub-model includes a diffraction coding compression module, an interference computing module and a diffraction decoding characterization module, wherein the output of the diffraction coding compression module is the input of the interference computing module, and the output of the interference computing module is the input of the diffraction decoding characterization module.

3. The large-scale reconfigurable photon matrix model according to claim 2 is characterized in that: The diffraction coding compression module is used to compress the first large matrix and the input vector into a second large matrix and a first vector; The interference calculation module is used to perform arbitrary matrix multiplication on the second largest matrix and the first vector to obtain a second vector; The diffraction decoding characterization module is used to calculate the second weight of diffraction decoding according to the second vector to obtain the output vector of each parallel channel.

4. The large-scale reconfigurable photon matrix model according to claim 3 is characterized in that: The interference calculation module is specifically used to: perform arbitrary matrix multiplication using an MZI or crossbar structure according to the second largest matrix and the first vector to obtain the second vector.

5. The large-scale reconfigurable photon matrix model according to claim 3 is characterized in that: The diffraction coding compression module is specifically used to: downsample the input vector into a first vector, and compress the first large matrix into a second large matrix through a first weight calculated by diffraction modulation.

6. The large-scale reconfigurable photon matrix model according to claim 1 is characterized in that: The integration module is specifically used to perform weighted summation on the output vectors of each parallel channel to obtain a target output vector.

7. A large-scale reconfigurable photon matrix architecture, characterized in that: include: At least one large-scale reconfigurable photonic matrix model as claimed in any one of claims 1 to 6.

8. The architecture according to claim 7, characterized in that include: A plurality of said mixed light calculation sub-models.

9. The architecture according to claim 8, characterized in that include: The multiple mixed light computing sub-models are connected in parallel.

10. The architecture according to claim 9, characterized in that The hybrid optical computing sub-model includes a diffraction coding compression module, an interference computing module and a diffraction decoding characterization module, wherein the output of the diffraction coding compression module is the input of the interference computing module, and the output of the interference computing module is the input of the diffraction decoding characterization module.

Images