Matrix calculation method and device based on large-scale reconfigurable photonic large matrix model

Abstract

The present disclosure relates to the field of optical computing technologies, and particularly to a large-scale reconfigurable photonic large matrix model and architecture. Among them, the model includes: a hybrid optical computing sub-model; the large-scale reconfigurable photonic large matrix model includes an acquisition module, a parallel input module, and an integration module. Among them, the acquisition module is used to acquire a first large matrix and an input vector that need to be calculated; the parallel input module is used to parallelly input the first large matrix and the input vector into the hybrid optical computing sub-model of each parallel channel based on the determined number of parallel channels, and obtain an output vector of each parallel channel; the integration module is used to integrate the output vectors of each parallel channel to obtain a target output vector. The present disclosure adopting the above solution realizes a new paradigm of large-scale general optical matrix operations while taking into account high computing accuracy and high data throughput, thereby supporting complex optical general computing tasks.

Description

Technical Field

[0001] The present disclosure relates to the field of optical computing technology, and in particular, to a large-scale reconfigurable photonic large matrix model and architecture. Background Art

[0002] With the rapid development of the fields of artificial intelligence and scientific computing, the complexity and scale of computing requirements are also increasing continuously. However, for existing large-scale matrix calculations, their performance is gradually approaching the saturation state, and it is difficult to effectively meet the increasingly stringent requirements for computing power and power consumption of large-scale complex algorithms. Light has natural advantages such as high throughput and low latency during the propagation process. Optical computing technology using photons instead of electrons as the computing carrier is regarded as the key to breaking the existing computing bottleneck. Summary of the Invention

[0003] The present disclosure aims to solve at least one of the technical problems in the related art to some extent.

[0004] To this end, the first object of the present disclosure is to propose a large-scale reconfigurable photonic large matrix model to achieve high-performance optical computing.

[0005] The second object of the present disclosure is to propose a large-scale reconfigurable photonic large matrix architecture.

[0006] To achieve the above object, an embodiment of the first aspect of the present disclosure proposes a large-scale reconfigurable photonic large matrix model, including: a hybrid optical computing sub-model; the large-scale reconfigurable photonic large matrix model includes an acquisition module, a parallel input module, and an integration module, where

[0007] The acquisition module is configured to acquire a first large matrix and an input vector that need to be calculated;

[0008] The parallel input module is configured to parallelly input the first large matrix and the input vector into the hybrid optical computing sub-model of each parallel channel based on the determined number of parallel channels, and obtain an output vector of each parallel channel;

[0009] The integration module is configured to integrate the output vectors of each parallel channel to obtain a target output vector.

[0010] Optionally, the hybrid optical computing sub-model includes a diffraction encoding and compression module, an interference calculation module, and a diffraction decoding and characterization module, where the output of the diffraction encoding and compression module is the input of the interference calculation module, and the output of the interference calculation module is the input of the diffraction decoding and characterization module.

[0011] Optionally, the diffraction encoding and compression module is configured to compress the first large matrix and the input vector into a second large matrix and a first vector;

[0012] The interference calculation module is configured to perform arbitrary matrix multiplication based on the second largest matrix and the first vector to obtain a second vector;

[0013] The diffraction decoding characterization module is configured to calculate the second weight of diffraction decoding based on the second vector to obtain the output vector of each parallel channel.

[0014] Optionally, the interference calculation module is specifically configured to: perform arbitrary matrix multiplication based on the second largest matrix and the first vector by using an MZI or crossbar structure to obtain a second vector.

[0015] Optionally, the diffraction encoding and compression module is specifically configured to: downsample the input vector to obtain a first vector, and compress the first largest matrix into a second largest matrix by using the first weight of diffraction modulation calculation.

[0016] Optionally, the integration module is specifically configured to: perform weighted summation on the output vectors of each parallel channel to obtain a target output vector.

[0017] To achieve the above object, an embodiment of the second aspect of the present disclosure provides a large-scale reconfigurable photonic large matrix architecture, including: at least one large-scale reconfigurable photonic large matrix model shown in any one of the foregoing first aspects.

[0018] Optionally, the architecture includes: a plurality of the hybrid optical computing sub-models.

[0019] Optionally, the connection manner between the plurality of the hybrid optical computing sub-models is in parallel.

[0020] Optionally, the hybrid optical computing sub-model includes a diffraction encoding and compression module, an interference calculation module, and a diffraction decoding characterization module, wherein the output of the diffraction encoding and compression module is the input of the interference calculation module, and the output of the interference calculation module is the input of the diffraction decoding characterization module.

[0021] In summary, the large-scale reconfigurable photonic large matrix model and architecture provided by the present disclosure, through a multi-channel parallel strategy, parallelize the first large matrix element array into the operation channels of multiple groups of finite-scale matrix element arrays, and realize the parameter scale adaptive precise fitting of arbitrary matrix multiplication operations through multiple hybrid optical computing sub-models. While taking into account high computing accuracy and high data throughput, a new paradigm of large-scale general optical matrix operations is realized, thereby supporting complex optical general computing tasks.

[0022] Additional aspects and advantages of the present disclosure will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present disclosure. Description of the Drawings

[0023] The above and / or additional aspects and advantages of the present disclosure will become apparent and be readily understood from the following description of embodiments in conjunction with the accompanying drawings, where:

[0024] Figure 1 FIG. 5 is a schematic structural diagram of a large-scale reconfigurable photonic large matrix model provided by an embodiment of the present disclosure;

[0025] Figure 2 FIG. 9 is a schematic structural diagram of a hybrid optical computing sub-model provided by an embodiment of the present disclosure. Specific Embodiments

[0026] Embodiments of the present disclosure will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present disclosure and should not be construed as limiting the present disclosure.

[0027] Currently, for most analog computing architectures, signals may attenuate during transmission and processing, and at the same time, noise will also affect the signal quality. As a result, errors or instability in the calculation results may occur. For optical computing models, due to factors such as material defects, wavefront errors, and non-uniform transmission in the optical system, various errors will inevitably be introduced in optical computing, and these errors may affect the accuracy of the calculation results, resulting in a deviation between the experimental results and the simulation data. If the number of parameters of the optical neural network is extended by simply stacking layers, the errors will gradually accumulate during propagation, ultimately having a huge error impact on the output.

[0028] In the prior art, an array composed of 56 cascaded Mach-Zehnder Interferometers (MZIs) can be used to complete the classification of four vowel acoustic signals, or a deep diffraction neural network based on the cascade of optical diffraction masks can be used to achieve visual tasks such as handwritten digit recognition and image saliency detection; the beam splitting and aggregation of the optical path endow the diffraction optical computing with the ability of multi-channel parallel processing.

[0029] However, the calculation effects in the above prior art are described by the classification / regression accuracy of a certain intelligent computing task, and cannot be measured by the precision deviation of a certain operation; moreover, it is difficult to cascade the calculation operators deeply in the above optical computing, which limits the representation ability and operation scale of optical computing.

[0030] The present disclosure will be described in detail below in conjunction with specific embodiments.

[0031] Figure 1The structural schematic diagram of a large-scale reconfigurable photonic large matrix model provided by an embodiment of the present disclosure. As Figure 1 shown, the large-scale reconfigurable photonic large matrix model includes: a hybrid optical computing sub-model; wherein, the large-scale reconfigurable photonic large matrix model includes an acquisition module, a parallel input module, and an integration module, wherein:

[0032] The acquisition module is configured to acquire a first large matrix and an input vector to be calculated;

[0033] The parallel input module is configured to parallelly input the first large matrix and the input vector into the hybrid optical computing sub-model of each parallel channel based on the determined number of parallel channels, and obtain an output vector of each parallel channel;

[0034] The integration module is configured to integrate the output vectors of each parallel channel to obtain a target output vector.

[0035] In an embodiment of the present disclosure, the above-mentioned large-scale reconfigurable photonic large matrix model is applicable to large matrix calculations. Based on this, the above-mentioned large-scale reconfigurable photonic large matrix model can be applicable to various scenarios, such as the classification of intelligent computing tasks or intelligent question-and-answer scenarios.

[0036] Among them, in an embodiment of the present disclosure, the large-scale reconfigurable photonic large matrix model can split the encoding of an ultra-large parameter matrix into multiple groups of parallel finite-scale parameter reconfigurable sub-matrices, and further decode and fuse the operation results of the sub-matrices into the result output of the final large matrix operation. And, in an embodiment of the present disclosure, an approximate fitting of any matrix multiplication operation can be achieved through a "diffraction-interference-diffraction" hybrid optical computing sub-model.

[0037] Specifically, in an embodiment of the present disclosure, the number of parallel channels can be determined according to the calculation accuracy requirement. Among them, the more the number of parallel channels, the higher the corresponding calculation accuracy. After determining the number of parallel channels based on the calculation accuracy requirement, each parallel channel corresponds to a hybrid optical computing sub-model, so as to construct an array of hybrid optical computing models, thereby realizing the parameter scale adaptive precise fitting of any matrix multiplication operation.

[0038] Among them, in an embodiment of the present disclosure, multiple parallel hybrid optical computing sub-models in the above-mentioned array of hybrid optical computing models are completed through joint training, and each hybrid optical computing sub-model corresponds to its own weight parameter.

[0039] Specifically, in an embodiment of the present disclosure, Figure 2 The structural schematic diagram of a hybrid optical computing sub-model provided by an embodiment of the present disclosure. As Figure 2As shown, the above-mentioned hybrid optical computing sub-model includes a diffraction encoding and compression module, an interference calculation module, and a diffraction decoding and characterization module. Among them, the output of the diffraction encoding and compression module is the input of the interference calculation module, and the output of the interference calculation module is the input of the diffraction decoding and characterization module.

[0040] In an embodiment of the present disclosure, the above-mentioned diffraction encoding and compression module is used to compress the first large matrix and the input vector into a second large matrix and a first vector; the interference calculation module is used to perform arbitrary matrix multiplication according to the second large matrix and the first vector to obtain a second vector; the diffraction decoding and characterization module is used to calculate the second weight of diffraction decoding according to the second vector to obtain the output vector of each parallel channel.

[0041] In an embodiment of the present disclosure, the above-mentioned diffraction encoding and compression module is specifically used to: downsample the input vector to a first vector, and compress the first large matrix into a second large matrix through the first weight of diffraction modulation calculation. Wherein, the dimension of the first vector is smaller than that of the input vector.

[0042] Exemplarily, in an embodiment of the present disclosure, assuming that the dimension of the input vector is N and the dimension of the first large matrix is N*N, the diffraction encoding and compression module can downsample the input vector to a first vector with a dimension of M (M < N), and compress the first large matrix with N channels into a second large matrix with M channels through the first weight of diffraction modulation calculation.

[0043] And, in an embodiment of the present disclosure, the above-mentioned interference calculation module is specifically used to: perform arbitrary matrix multiplication according to the second large matrix and the first vector by using an MZI or crossbar structure to obtain a second vector.

[0044] Exemplarily, in an embodiment of the present disclosure, the interference calculation module can perform arbitrary matrix multiplication of M*M according to the second large matrix and the first vector by using an MZI or crossbar structure to obtain a second vector. Wherein, the dimension of the second vector is 1×M.

[0045] Furthermore, in an embodiment of the present disclosure, when the above-mentioned diffraction decoding and characterization module calculates the second weight of diffraction decoding according to the second vector to obtain the output vector of each parallel channel, it can calculate the second weight of diffraction decoding on the M-dimensional second vector output by the interference calculation module and restore it to an N-dimensional output vector, that is, the dimension of the output vector is the same as that of the input vector.

[0046] In addition, in an embodiment of the present disclosure, 100k groups of arbitrary input vectors can be selected, and 100k groups of true value operation outputs corresponding to the input vectors can be obtained through the operation of the to-be-fitted N*N dimensional matrix, so as to obtain the data set required for training the above-mentioned hybrid optical computing model array, and the hybrid optical computing model array can be trained by using this data set to obtain the weight parameters in each hybrid optical computing sub-model, so that the effect of optical computing approaches the operation effect of the actual to-be-fitted matrix.

[0047] Further, in an embodiment of the present disclosure, the above-mentioned integration module is specifically configured to: perform weighted summation on the output vectors of each parallel channel to obtain a target output vector.

[0048] In an embodiment of the present disclosure, the dimension of the large-scale matrix to be calculated is N*N, and the joint optimization calculation of multiple M*M parallel small matrices can be used to obtain the required results within the allowable error range. Among them, in the current large matrix operation based on Transformer, the matrix to be calculated generally has strong sparsity (that is, there is a large parameter redundancy). Using the above-mentioned large-scale reconfigurable photonic large matrix model can effectively reduce the number of multiplication and addition calculations in matrix operations, making the operation more efficient. At the same time, the above-mentioned method of splitting matrix operations can theoretically enable small-scale computing hardware to support the calculation of larger matrices through reuse, and has strong scalability.

[0049] In summary, the model provided in this embodiment, through the multi-channel parallel strategy, parallelizes the first large matrix element array into the operation channels of multiple groups of finite-scale matrix element arrays, and realizes the parameter scale adaptive precise fitting of arbitrary matrix multiplication operations through multiple hybrid optical computing sub-models. While taking into account high computing accuracy and high data throughput, a new paradigm for large-scale general optical matrix operations is realized, which can thus support complex optical general computing tasks, has a wide range of application prospects, and can be applied to fields such as unmanned systems, autonomous driving, and ultrafast science.

[0050] To implement the above embodiment, the present disclosure also proposes a large-scale reconfigurable photonic large matrix architecture, including: at least one large-scale reconfigurable photonic large matrix model provided in the foregoing embodiment.

[0051] Optionally, the architecture includes: multiple hybrid optical computing sub-models. Among them, the connection mode between multiple hybrid optical computing sub-models is parallel.

[0052] Optionally, the hybrid optical computing sub-model includes a diffraction encoding and compression module, an interference calculation module, and a diffraction decoding and characterization module. Among them, the output of the diffraction encoding and compression module is the input of the interference calculation module, and the output of the interference calculation module is the input of the diffraction decoding and characterization module.

[0053] The collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved in this disclosure all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0054] It should be noted that personal information from users should be collected for legal and reasonable purposes and not shared or sold outside of these legal uses. In addition, such collection / sharing should be carried out after obtaining the informed consent of the user, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization including authorizing relevant user information before the user uses the function. In addition, any necessary steps should be taken to safeguard and protect access to such personal information data and ensure that others with access to the personal information data comply with their privacy policies and procedures.

[0055] This disclosure anticipates providing embodiments that allow users to selectively block the use or access of personal information data. That is, this disclosure anticipates providing hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, the risk can be minimized by restricting data collection and deleting the data. In addition, when applicable, personal identifiers are removed from such personal information to protect the privacy of the user.

[0056] The acquisition, transmission, storage, use, processing, etc. of data in the technical solutions of this disclosure all comply with the relevant provisions of national laws and regulations.

[0057] It should be noted that in the embodiments of this disclosure, certain industry-existing solutions such as certain software, components, models, etc. may be mentioned. They should be considered exemplary, and their purpose is only to illustrate the feasibility in the implementation of the technical solutions of this application, but it does not mean that the applicant has already or necessarily used this solution.

[0058] In the descriptions of the foregoing embodiments, the descriptions referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0059] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the technical features indicated. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In the description of the present disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0060] Any process or method description represented in a flowchart or otherwise described herein may be understood to represent code including one or more modules, segments, or portions of executable instructions for implementing a customized logic function or process. The scope of the preferred embodiments of the present disclosure includes additional implementations, where functions may be executed in a substantially simultaneous manner or in a reverse order according to the functions involved, rather than in the order shown or discussed. This should be understood by those skilled in the art to which the embodiments of the present disclosure pertain.

[0061] The logic and / or steps represented in a flowchart or otherwise described herein, for example, may be considered as a sequenced list of executable instructions for implementing a logical function, and may be specifically implemented in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device. For the purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of the computer-readable medium include the following: an electrical connection portion having one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium may even be paper or other suitable media on which a program can be printed, as the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other appropriate processing as necessary, and then stored in a computer memory.

[0062] It should be understood that various parts of the present disclosure can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following techniques known in the art or a combination thereof can be used: discrete logic circuits having logic gate circuits for implementing logic functions on data signals, application-specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0063] Those of ordinary skill in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0064] In addition, in each of the embodiments of the present disclosure, the functional units can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0065] The above-mentioned storage medium can be a read-only memory, a magnetic disk, an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A matrix calculation method based on a large-scale reconfigurable photon matrix model, characterized in that: include: Hybrid optical computing sub-model; The large-scale reconfigurable photon matrix model includes an acquisition module, a parallel input module and an integration module, wherein: Acquire the first largest matrix and input vector to be calculated through the acquisition module; Through the parallel input module, based on the determined number of parallel channels, the first large matrix and the input vector are input in parallel into the mixed light calculation sub-model of each parallel channel to obtain an output vector of each parallel channel; The output vectors of each parallel channel are integrated by the integration module to obtain a target output vector. The hybrid optical computing sub-model includes a diffraction coding compression module, an interference computing module and a diffraction decoding characterization module, wherein the output of the diffraction coding compression module is the input of the interference computing module, and the output of the interference computing module is the input of the diffraction decoding characterization module; The method further comprises: compressing the first large matrix and the input vector into a second large matrix and a first vector by the diffraction coding compression module; Performing arbitrary matrix multiplication on the second largest matrix and the first vector by the interference calculation module to obtain a second vector; The diffraction decoding characterization module calculates a second weight of diffraction decoding according to the second vector to obtain an output vector of each parallel channel.

2. The method according to claim 1, characterized in that Also includes: The interference calculation module performs arbitrary matrix multiplication according to the second largest matrix and the first vector using an MZI or crossbar structure to obtain a second vector.

3. The method according to claim 1, characterized in that Also includes: The input vector is downsampled to a first vector by the diffraction coding compression module, and the first large matrix is ​​compressed to a second large matrix by a first weight calculated by diffraction modulation.

4. The method according to claim 1, characterized in that: Also includes: The output vectors of each parallel channel are weighted and summed by the integration module to obtain a target output vector.

5. A matrix computing device based on a large-scale reconfigurable photon matrix model, characterized in that: include: The hybrid optical computing sub-model, the large-scale reconfigurable photon matrix model includes an acquisition module, a parallel input module and an integration module, wherein: The acquisition module is used to acquire the first largest matrix and input vector to be calculated; The parallel input module is used to input the first large matrix and the input vector in parallel into the mixed light calculation sub-model of each parallel channel based on the determined number of parallel channels, so as to obtain the output vector of each parallel channel; The integration module is used to integrate the output vectors of each parallel channel to obtain a target output vector. The hybrid optical computing sub-model includes a diffraction coding compression module, an interference computing module and a diffraction decoding characterization module, wherein the output of the diffraction coding compression module is the input of the interference computing module, and the output of the interference computing module is the input of the diffraction decoding characterization module; The diffraction coding compression module is used to compress the first large matrix and the input vector into a second large matrix and a first vector; The interference calculation module is used to perform arbitrary matrix multiplication on the second largest matrix and the first vector to obtain a second vector; The diffraction decoding characterization module is used to calculate the second weight of diffraction decoding according to the second vector to obtain the output vector of each parallel channel.

6. The device according to claim 5, characterized in that include: A plurality of said mixed light calculation sub-models.

7. The device according to claim 6, characterized in that The multiple mixed light computing sub-models are connected in parallel.

Images