An optical computing device, system, and method of computation

Abstract

An optical computing device, system and method. The optical computing device includes a linear operation module, a first delay module and a coupler. The linear operation module can receive a first set of optical signals and a first electrical signal, and modulate the first set of optical signals according to the first electrical signal to output a second set of optical signals. The linear operation module can subsequently receive a third set of optical signals and a second electrical signal, and modulate the third set of optical signals according to the second electrical signal to output a fourth set of optical signals. The first delay module can adjust the time delay of the second set of optical signals, and the coupler combines the delayed second set of optical signals and the fourth set of optical signals into a fifth set of optical signals, which indicates a first calculation result of the multiplication and accumulation operation of the first set of data and the first part of weights and the second part of weights. For the optical computing system, different multiplication and accumulation operations can be realized by adjusting the input optical signals and electrical signals, ensuring the efficiency of the optical computing system.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to an optical computing device, system, and computing method. Background Technology

[0002] Neural network models, such as convolutional and fully connected layers, typically require extensive data computation and storage of process data. To improve the computational speed of neural network models, optical computing systems can be used to perform the data computation process and obtain the final results.

[0003] Optical computing systems can be categorized into two types based on their implementation: spatial optical computing systems and on-chip optical computing systems. Spatial optical computing systems primarily utilize 4F optical systems, relying on Fourier optical transforms to perform convolution operations and simulate neural network models such as convolutional neural networks (CNNs). However, these systems are disadvantageous in terms of speed and size. On-chip optical computing systems, on the other hand, are based on silicon-based photonic integration technology. They mainly utilize optical devices such as Mach-Zehnder interferometers (MZIs) to construct optical multipliers and adders to achieve convolution operations. However, these systems include a limited number of MZIs, requiring multiple multiplication-addition iterations to achieve the operation. Furthermore, on-chip optical computing systems use heated electrodes to adjust the phase of the optical signal, making them prone to thermal crosstalk, which reduces computational accuracy and overall system efficiency. Summary of the Invention

[0004] This application provides an optical computing device, system, and computing method to provide a highly efficient optical computing system.

[0005] In a first aspect, embodiments of this application provide an optical computing device, which includes a linear operation module, a first delay module, and a coupler.

[0006] The linear operation module can receive a first set of optical signals to indicate a first set of data, and can also receive a first electrical signal to indicate a first portion of the weight values ​​in the first weights. The linear operation module modulates the first set of optical signals according to the first electrical signal to output a second set of optical signals, wherein the second set of optical signals is used to indicate the calculation result of the first set of data and the first portion of the weight values, and the first weights are the weights of the first neural network layer.

[0007] The linear operation module can subsequently receive a third set of optical signals to indicate the first set of data, and a second electrical signal to indicate the second part of the weight value in the first weight. The linear operation module modulates the third set of optical signals according to the second electrical signal to output a fourth set of optical signals, wherein the fourth set of optical signals is used to indicate the calculation result of the first set of data and the second part of the weight value.

[0008] The first delay module can adjust the delay of the second set of optical signals; then, the coupler can merge the delayed second set of optical signals with the fourth set of optical signals into a fifth set of optical signals, wherein the fifth set of optical signals indicates the first calculation result after multiplying and adding the first set of data with the first part of the weights and the second part of the weights.

[0009] The aforementioned optical computing device allows the linear arithmetic unit to receive multiple sets of optical and electrical signals sequentially, and in conjunction with a coupler and a first delay module, output a set of optical signals indicating the results of multiplication-addition operations. In this optical computing device, different multiplication-addition operations can be achieved simply by adjusting the input optical and electrical signals. Furthermore, the device does not require heating electrodes to adjust the phase of the optical signals, ensuring computational accuracy and thus guaranteeing the efficiency of the optical computing device.

[0010] In one possible design, the optical computing device may also include a filtering module. Besides multiply-accumulate operations, the optical computing device can also implement pooling operations using the filtering module. The following example illustrates the pooling operation performed by the optical computing device on the sixth group of optical signals. The linear operation module can receive the sixth group of optical signals and a third electrical signal indicating the second weights. The sixth group of optical signals indicates the second calculation result after multiplying and adding the first group of data with the first weights. The second calculation result may include the first calculation result, and the second weights are the weights of the second neural network layer.

[0011] The linear operation module modulates the sixth group of optical signals according to the third electrical signal to output the seventh group of optical signals. The filtering module can receive the seventh group of optical signals, filter out invalid signals in the seventh group of optical signals, and output the eighth group of optical signals. The eighth group of optical signals is used to indicate the calculation result of the pooling operation on the second calculation result.

[0012] The optical computing device described above can not only perform multiplication and addition operations but also pooling operations, thus improving the versatility of the optical computing device.

[0013] In one possible design, where the optical computing device includes a filtering module, the optical computing device can also perform convolution operations using the filtering module. This will be illustrated using the example of the optical computing device performing a convolution operation on the ninth group of optical signals.

[0014] The linear operation module can receive the ninth set of optical signals and the fourth electrical signal indicating the third weight. The ninth set of optical signals is used to indicate the second set of data.

[0015] The linear operation module can modulate the ninth group of optical signals according to the fourth electrical signal to output the tenth group of optical signals. The tenth group of optical signals is used to indicate the third calculation result after the second group of data and the third weight are multiplied and added. The third weight is the weight of the third neural network layer.

[0016] The filtering module receives the tenth group of optical signals, filters out invalid signals in the tenth group of optical signals, and outputs the eleventh group of optical signals. The eleventh group of optical signals is used to indicate the calculation result of the convolution operation on the third calculation result.

[0017] The optical computing device described above can not only perform multiplication and addition operations, but also convolution and pooling operations, further improving the versatility of the optical computing device and enabling it to perform operations on a wide range of neural network models.

[0018] In one possible design, the optical computing device may also include a nonlinear operation module to implement the activation function configured in the neural network model. The nonlinear operation module can perform nonlinear operations on the signals output by the coupler, such as the sixth, fifth, eighth, or eleventh optical signals; these nonlinear operations are activation function operations.

[0019] The optical computing device described above not only performs multiplication-accumulation, convolution, and pooling operations, but also performs activation function operations on the calculation results of these operations, ensuring the versatility of the optical computing device.

[0020] In one possible design, the optical computing device may also include a dual-mode beamsplitter capable of splitting a received set of optical signals indicating a set of data into two identical sets of optical signals indicating the same set of data. For example, the dual-mode beamsplitter may receive a twelfth set of optical signals indicating the first set of data and split the twelfth set of optical signals into a first set of optical signals and a third set of optical signals.

[0021] With the aforementioned optical computing device, the dual-mode beam splitter in the optical computing device eliminates the need to input a set of optical signals indicating the same data multiple times. Instead, it only requires inputting one set of optical signals to split multiple sets of optical signals indicating the same data, which can effectively improve the computing efficiency and performance of the optical computing device.

[0022] In one possible design, the optical computing device further includes a second delay module. The dual-mode beam splitter can transmit the delayed second set of optical signals to the second delay module. The second delay module can adjust the time delay of the second set of optical signals so that the second set of optical signals is transmitted to the coupler simultaneously with the fourth set of optical signals.

[0023] With the aforementioned optical computing device, the coupler can simultaneously receive multiple sets of optical signals and can effectively merge them into a single set of optical signals.

[0024] In one possible design, the linear operation module includes N photoelectric modulators.

[0025] Any one of the N photoelectric modulators can receive one optical signal from the first group of optical signals and one electrical signal from the first electrical signal used to indicate the first part of the weight, and modulate the received optical signal according to the received electrical signal; similarly, any one of the photoelectric modulators can also receive one optical signal from the third group of optical signals and one electrical signal from the second electrical signal used to indicate the second part of the weight, and modulate the received optical signal according to the received electrical signal.

[0026] With the aforementioned optical computing device, the linear operation module can effectively perform multiplication and addition operations using N photoelectric modulators, ensuring the high efficiency of the optical computing device.

[0027] In one possible design, the linear operation module also includes N phase modulators and N-1 phase-locked loop modules;

[0028] A phase modulator can be connected to a photoelectric modulator, and the phase modulator can adjust the phase of the first or second set of optical signals after being modulated by the modulator; a phase-locked module can be connected to two of the N phase modulators to lock the phase of the first or second set of optical signals after being modulated by the two phase modulators.

[0029] The phase of the optical signal can be adjusted by the phase modulator through the optical computing device, thereby adjusting the sign of the data indicated by the optical signal. The phase-locked loop module can ensure that the phase modulator can effectively adjust the phase of the optical signal.

[0030] In one possible design, the linear operation module also includes an amplifier that can amplify the power of the fifth, sixth, eighth, or eleventh optical signals.

[0031] The optical computing device described above can reduce the loss of optical signals in the optical computing device by using an amplifier.

[0032] Secondly, this application provides a calculation method, the beneficial effects of which can be found in the relevant description of the first aspect, and will not be repeated here. This method is executed by an optical computing device, and includes:

[0033] The linear operation module receives a first set of optical signals and a first set of electrical signals, wherein the first set of optical signals is used to indicate a first set of data and the first set of electrical signals is used to indicate a first part of the weight value in the first weight; the first set of optical signals is modulated according to the first set of electrical signals to output a second set of optical signals, wherein the second set of optical signals is used to indicate the calculation result of the first set of data and the first part of the weight value, and the first weight is the weight of the first neural network layer;

[0034] The linear operation module receives a third set of optical signals and a second set of electrical signals. The third set of optical signals is used to indicate the first set of data, and the second set of electrical signals is used to indicate the second part of the weight value in the first weight. The module modulates the third set of optical signals according to the second set of electrical signals to output a fourth set of optical signals. The fourth set of optical signals is used to indicate the calculation result of the first set of data and the second part of the weight value.

[0035] The first delay module adjusts the delay of the second set of optical signals.

[0036] The coupler combines the delayed second group of optical signals with the fourth group of optical signals into a fifth group of optical signals. The fifth group of optical signals indicates the first calculation result after multiplying and adding the first group of data with the first part of the weights and the second part of the weights.

[0037] In one possible design, the linear operation module receives a sixth set of optical signals, which are used to indicate the second calculation result after the first set of data and the first weight are multiplied and added, and a third electrical signal used to indicate the second weight. The second calculation result includes the first calculation result. The second weight is the weight of the second neural network layer.

[0038] The linear operation module modulates the sixth group of optical signals according to the third electrical signal to output the seventh group of optical signals.

[0039] The filtering module receives the seventh set of optical signals, filters out invalid signals in the seventh set of optical signals, and outputs the eighth set of optical signals. The eighth set of optical signals is used to indicate the calculation result of the pooling operation on the second calculation result.

[0040] In one possible design, the linear operation module receives a ninth set of optical signals, which are used to indicate the second set of data.

[0041] The linear operation module receives a fourth electrical signal used to indicate the third weight, and modulates the ninth group of optical signals according to the fourth electrical signal to output a tenth group of optical signals. The tenth group of optical signals is used to indicate the third calculation result after the second group of data is multiplied and added with the third weight. The third weight is the weight of the third neural network layer.

[0042] The filtering module receives the tenth group of optical signals, filters out invalid signals in the tenth group of optical signals, and outputs the eleventh group of optical signals. The eleventh group of optical signals is used to indicate the calculation result of the convolution operation on the third calculation result.

[0043] Thirdly, this application provides an optical computing system, wherein the optical computing device may include a processor and an optical computing apparatus as described in the first aspect or any possible aspect thereof, wherein the processor is configured to transmit data to be computed to the optical computing apparatus, such as a first set of data or a second set of data, and may also transmit a first weight or a second weight.

[0044] Fourthly, this application provides a computer-readable storage medium that, when executed by a computing device, performs the methods provided in the second aspect or any possible design of the second aspect. The storage medium stores a program. The storage medium includes, but is not limited to, volatile memory, such as random access memory, and non-volatile memory, such as flash memory, hard disk drive (HDD), and solid-state drive (SSD).

[0045] Fifthly, this application provides a computing device program product comprising computer instructions that, when executed by a computing device, enable the computing device to perform the methods provided in the second aspect or any possible design of the second aspect. The computer program product can be a software installation package, which can be downloaded and executed on a computing device when the methods provided in the second aspect or any possible design of the second aspect are required. Attached Figure Description

[0046] Figure 1 A schematic diagram of the structure of an optical computing device provided in this application;

[0047] Figure 2 A schematic diagram of an optical signal transmission path provided in this application;

[0048] Figure 3a A schematic diagram of multiplication and addition operation provided in this application;

[0049] Figure 3bA schematic diagram of a first part weight value and a second part weight value provided for this application;

[0050] Figure 4a A schematic diagram of the structure of an optical computing device provided in this application;

[0051] Figure 4b A schematic diagram of the structure of an optical computing device provided in this application;

[0052] Figure 5 A schematic diagram of the structure of an optical computing device provided in this application;

[0053] Figure 6a A schematic diagram of the structure of an optical computing device provided in this application;

[0054] Figure 6b A schematic diagram of the structure of an optical computing device provided in this application;

[0055] Figure 7 A schematic diagram of the structure of an optical computing device provided in this application;

[0056] Figure 8a A schematic diagram of an optical signal transmission path provided in this application;

[0057] Figure 8b A schematic diagram of an optical signal transmission path provided in this application;

[0058] Figure 9a A schematic diagram of the structure of a linear operation module provided in this application;

[0059] Figure 9b A schematic diagram of the structure of a linear operation module provided in this application;

[0060] Figure 10 This application provides a schematic diagram of the structure of a first delay module;

[0061] Figure 11 A schematic diagram of the structure of a filtering module provided in this application;

[0062] Figure 12 This application provides a schematic diagram of the structure of a dual-mode beam splitter.

[0063] Figure 13 A schematic diagram of multiplication and addition operation provided in this application;

[0064] Figure 14 A schematic diagram of a convolution operation provided in this application;

[0065] Figure 15aA schematic diagram of the input serial optical signal and the output optical signal during the first linear operation process provided in this application;

[0066] Figure 15b A schematic diagram of the input serial optical signal and the output optical signal in a second linear operation process provided in this application;

[0067] Figure 15c A schematic diagram of an optical signal filtered by a filtering module provided in this application;

[0068] Figure 15d A schematic diagram of the optical signal output by an optical computing device provided in this application;

[0069] Figure 16 A schematic diagram of a pooling operation is provided for this application;

[0070] Figure 17 A schematic diagram of the input serial optical signal and the output optical signal in a linear operation process provided in this application;

[0071] Figure 18 A calculation method provided for this application;

[0072] Figure 19 This application provides an optical computing system. Detailed Implementation

[0073] like Figure 1 As shown, this application provides an optical computing device, which includes a linear operation module 100, a first delay module 200, and a coupler 300. This application does not limit the specific form of the optical computing device; for example, the optical computing device can be an optical computing chip.

[0074] In this embodiment of the application, the linear operation module 100 can modulate the optical signal input to the linear operation module 100 according to the received electrical signal. The first delay module 200 can adjust the delay of the optical signal output by the linear operation module 100. After the first delay module 200 adjusts the delay of the optical signal output by the linear operation module 100, the coupler 300 can merge multiple sets of optical signals output by the linear operation module 100 in sequence and output a set of optical signals to indicate the calculation result of the multiplication and addition operation of a set of data and weights.

[0075] The following example illustrates the computational process in an optical computing device by having the linear processing module 100 receive two sets of optical signals, namely the first set and the third set:

[0076] First, while receiving the first set of optical signals indicating the first set of data, the linear operation module 100 can also receive a first electrical signal indicating the first part of the weight values ​​in the first weight. The linear operation module 100 can modulate the first set of optical signals according to the first electrical signal, modulating the light intensity of the first set of optical signals, and output a second set of optical signals. The output second set of optical signals can indicate the calculation result of the first set of data and the first part of the weight values. The first weight is the weight set on the first neural network layer.

[0077] Since the linear operation module 100 can only perform a limited amount of computation on a single optical signal in one linear operation, it can only output an optical signal indicating the multiplication and addition operation of the first set of data and a portion of the weights (such as the first part of the weight value). If the linear operation module 100 needs to complete all the computations, it needs to continue receiving the next set of optical signals for linear operation so that the coupler 300 can subsequently combine multiple sets of different optical signals output by the linear operation module 100 and output a single optical signal indicating the calculation result of the multiplication and addition operation between the first set of data and the first weight.

[0078] To distinguish it from the first group of optical signals, the optical signal subsequently received by the linear operation module 100 to indicate the first group of data is taken as the third group of optical signals.

[0079] During the process of receiving the third set of optical signals, the linear operation module 100 can also receive a second electrical signal used to indicate the second part of the weight value in the first weight. The linear operation module 100 can modulate the third set of optical signals according to the second electrical signal, modulate the light intensity of the third set of optical signals, and output a fourth set of optical signals. The output fourth set of optical signals can indicate the calculation result of the first set of data and the second part of the weight value.

[0080] As can be seen from the above, the linear operation module 100 outputs the second set of optical signals and the fourth set of optical signals in succession. There is a time delay between these two optical signals, so they cannot be directly merged.

[0081] Therefore, the first delay module 200 can adjust the delay of the second set of optical signals; then, after the coupler 300 receives the fourth set of optical signals and the second set of optical signals with adjusted delay, it can merge the second set of optical signals and the delayed fourth set of optical signals into a fifth set of optical signals. The fifth set of optical signals is used to indicate the calculation result after the first set of data is multiplied and added with the first part of the weight and the second part of the weight.

[0082] The transmission paths of the second and fourth groups of optical signals in the optical computing device are described below.

[0083] See Figure 2 , Figure 2The transmission path 1 of the second set of optical signals after being output from the linear processing module 100 is marked with a solid black line. Figure 2 It can be seen that after the second set of optical signals is output from the linear operation module 100, it is transmitted to the first delay module 200, and then output from the first delay module 200 to the coupler 300.

[0084] Figure 2 The transmission path 2 of the fourth group of optical signals after being output from the linear processing module 100 is marked with a black dashed line. Figure 2 As can be seen, the fourth set of optical signals can be directly transmitted to the coupler 300 after being output from the linear operation module 100.

[0085] In such Figures 1-2 In the optical computing device shown, the second set of optical signals can indicate the calculation result after the first set of data and the first part of the weight values ​​are multiplied and added. The fourth set of optical signals can indicate the result after the first set of data and the second part of the weight values ​​are multiplied and added. After the delay of the second set of optical signals is adjusted by the first delay module 200, the coupler 300 couples the delayed second set of optical signals and the fourth set of optical signals into a fifth set of optical signals, which also completes the multiplication and addition operation between the first set of data and the first part of the weights, as well as the second part of the weights, and realizes the operation process of the fully connected layer.

[0086] The following describes the operations performed by the fully connected layer in the optical computing device. See [link to documentation]. Figure 3a , Figure 3a The left side shows a multiply-accumulate operation required by a fully connected layer, which can be converted to a more intuitive form. Figure 3a The array shown on the right is in the form of a 4x4 linear operation, requiring four input data points (X...). c1 X c2 X c3 X c4 ), four output data (X) n1 X n2 X n3 X n4 The required weight is (W) 11 ~W 44 If the linear operation module 100 can only perform two different modulations on a single optical signal at a time (i.e., the linear operation module 100 only includes two photoelectric modulators), then... Figure 3a As can be seen from the multiplication and addition operations, for any input data, four different adjustments are required, with X... c1 For example, for X c1 Four different adjustments need to be performed to obtain W. 11 X c1 W 12 X c1 W13 X c1 W 14 X c1 In order to complete such Figure 3a The multiplication and addition operation shown requires the linear operation module 100 to receive two sets of instructions X sequentially. c1 X c2 X c3 X c4 The optical signal, such as if it were the first group of optical signals and the third group of optical signals, also needs to receive, sequentially, a first electrical signal indicating the first part of the weight value and a second electrical signal indicating the second part of the weight value. See [link to documentation]. Figure 3b The solid box represents the first part of the weight values, and the dashed box represents the second part of the weight values. This allows the system to work in conjunction with the first delay module 200 and the coupler 300 to output the indicator X. n1 X n2 X n3 X n4 The fifth group of optical signals.

[0087] It should be noted that the above-mentioned optical computing device is only illustrated by the example of the linear operation module 100 receiving two sets of optical signals indicating the first set of data. If the first weight has a third weight value in addition to the first and second weight values, the linear operation module 100 can continue to receive a set of optical signals indicating the first set of data. The linear operation module 100 modulates the set of optical signals according to the electrical signal indicating the third weight value and outputs a modulated set of optical signals. The first delay module 200 can delay the fifth set of optical signals, and the coupler 300 can merge the fifth set of optical signals and the modulated set of optical signals into a single set of optical signals. Similarly, if the first weight also includes a fourth weight value, the optical computing device can also adopt a similar process as described above. The linear operation module 100 continues to receive a set of optical signals indicating the first set of data and modulates the set of optical signals. The first delay module 200 delays the previously merged set of optical signals. The coupler 300 merges the set of optical signals delayed by the first delay module 200 with the set of optical signals modulated by the linear operation module 100 this time. The above process is executed repeatedly until the multiplication and addition operation between the first set of data and the first weight is completed. In this embodiment, the sixth set of optical signals is used to indicate the calculation result of the multiplication and addition operation between the first set of data and the first weight.

[0088] Optical computing devices can also perform operations on other neural network models, such as convolution operations in convolutional layers and pooling operations in pooling layers. Since some unnecessary data, i.e., invalid data, needs to be removed during convolution or pooling operations, the optical computing device may also include a filtering module 400. For example... Figure 4aAs shown, a filtering module 400 can be set on the transmission path of the second set of optical signals. This filtering module 400 can filter out some optical signals that do not need to perform operations, i.e., invalid signals, from the set of optical signals. These invalid signals indicate invalid data. After filtering out some optical signals that do not need to perform operations, the remaining optical signals in the set of optical signals are transmitted to the first delay module 200 along the transmission path of the second set of optical signals.

[0089] The following example illustrates the transmission process of optical signals in an optical computing device when the device performs pooling operations, using the example of two sets of optical signals received by the linear operation module 100 in the optical computing device, namely the first set of optical signals or the third set of optical signals.

[0090] First, when the coupler 300 outputs the sixth group of optical signals (the generation process of the sixth group of optical signals can be found in the above content, and will not be repeated here), the sixth group of optical signals is transmitted to the linear operation module 100 through the first delay module 200 and the filtering module 300.

[0091] The linear operation module 100 modulates the sixth group of optical signals according to the received third electrical signal indicating the second weight, and outputs the seventh group of optical signals. The second weight is the weight set on the second neural network layer (such as a pooling layer).

[0092] Subsequently, the seventh group of optical signals is transmitted to the filtering module 400 through the first delay module 200 (the first delay module 200 does not need to adjust the delay of the second group of optical signals), and the sixth group of optical signals is transmitted to the filtering module 400. The filtering module 400 filters out invalid signals from the seventh group of optical signals. Invalid signals refer to invalid data that needs to be deleted in the pooling operation. The remaining optical signals in the seventh group of optical signals are output. For ease of explanation, the eighth group of optical signals is used to indicate the remaining optical signals in the seventh group of optical signals. The eighth group of optical signals can indicate the final result of the pooling operation on the calculation result of the multiplication and addition operation between the first group of data and the first weight.

[0093] The following explanation uses the implementation of convolution operations in an optical computing device as an example. Distinguishing itself from the aforementioned groups of optical signals and electrical signals, this explanation uses the group of optical signals received by the linear operation module 100 as the ninth group of optical signals, and the received electrical signal as the fourth optical signal indicating the third weight. Here, the third weight refers to the weights set on the third neural network layer (such as a convolutional layer).

[0094] The linear operation module 100 modulates the ninth group of optical signals according to the received fourth electrical signal, modulates the light intensity of the ninth group of optical signals, and outputs the tenth group of optical signals. The output tenth group of optical signals can indicate the calculation results of the second group of data and the third weight.

[0095] It should be noted that this explanation only uses the example of the linear operation module 100 receiving one set of optical signals and outputting the tenth set of optical signals. In actual operation, similar to the way the linear operation module 100 cooperates with the first delay module 200 and the coupler 300 to output the sixth set of optical signals, the linear operation module 100 can also receive multiple sets of optical signals in sequence and cooperate with the first delay module 200 and the coupler 300 to perform linear operations repeatedly to obtain the tenth set of optical signals.

[0096] The tenth group of optical signals is transmitted to the filtering module 400 through the first delay module 200 (the first delay module 200 may not adjust the delay of the tenth group of optical signals). The filtering module 400 filters out invalid signals from the tenth group of optical signals. Invalid signals refer to invalid data that needs to be deleted in the pooling operation. The remaining optical signals in the tenth group of optical signals are output. For ease of explanation, the eleventh group of optical signals is used to indicate the remaining optical signals in the tenth group of optical signals. The eleventh group of optical signals can indicate the final result of the convolution operation on the second group of data.

[0097] It should be noted that the first neural network layer, the second neural network layer, and the third neural network layer mentioned above can be any neural network layer in any neural network model, and the first neural network layer and the second neural network layer can also be two adjacent neural network layers in a neural network model.

[0098] Typically, optical signals travel at different times along different transmission paths. To ensure that the second and fourth sets of optical signals are transmitted to the coupler 300 simultaneously, the optical computing device may also include a second delay module 500, such as... Figure 4b As shown, the second delay module 500 can be placed on the transmission path of the second set of optical signals to further adjust the delay of the second set of optical signals.

[0099] As one possible implementation method, such as Figure 5 As shown, the control system may also include a dual-mode beam splitter 600, which can receive a first set of optical signals and a third set of optical signals and transmit the first set of optical signals and the third set of optical signals to the linear operation module 100. The dual-mode beam splitter 600 can also receive a second set of optical signals and transmit the second set of optical signals to the second delay module 500 or the first delay module 200.

[0100] It should be noted that, in addition to transmitting a set of optical signals to the linear operation module 100, the second delay module 500, or the first delay module 200, the dual-mode beam splitter 600 can also split a set of optical signals into two sets of optical signals identical to the original set. For example, the dual-mode beam splitter 600 can first receive the twelfth set of optical signals and then split it into a first set and a third set of optical signals.

[0101] As one possible implementation method, such as Figure 6a As shown, the optical computing device may also include a nonlinear operation module 700, which can perform nonlinear operations on the second set of optical signals to realize the function of the activation function configured in the neural network model.

[0102] It should be noted that the embodiments of this application do not limit the location of the nonlinear operation module 700. For example, the nonlinear operation module 700 can be connected to the coupler 300 to receive the fifth, sixth, eighth, and eleventh optical signals from the coupler 300, or it can be connected to the first delay module 200, so that the fifth or sixth, eighth, and eleventh optical signals can be received through the first delay module 200. Figure 6a As shown.

[0103] As one possible implementation method, such as Figure 6b As shown, the optical computing device may also include an optical amplifier 1000, which can be coupled to a coupler 300 (e.g., Figure 6b The amplifier 1000 can be a semiconductor optical amplifier (SOA) or an erbium-doped fiber application amplifier (EDFA). It can also be a nonlinear operation unit 700, which amplifies the power of the fifth, sixth, eighth, and eleventh optical signals.

[0104] In actual computation, not all computations require the participation of the nonlinear computation module 700. To improve the versatility of the optical computing device and make it suitable for various computing scenarios, the optical computing device may include multiple optical switches to change the transmission direction of optical signals within the device. Optionally, the optical computing device may also include multiple couplers to combine optical signals output from different modules or devices.

[0105] See Figure 7This application provides an optical computing device, including a dual-mode beam splitter 600, a linear operation module 100, a second delay module 500, a first delay module 200, a nonlinear operation module 700, and a filtering module 400. The optical computing device also includes three couplers: a first coupler 810, a second coupler 820, and a third coupler 830. It further includes two optical switches: a first optical switch 910 and a second optical switch 920. An amplifier 1000 is also included, connected to either the third coupler 830 or the first optical switch 910. It should be noted that this application does not limit the location of the amplifier 1000; the amplifier 1000 can be located at any position along the path: second coupler 820 -> third coupler 830 -> first optical switch 910 -> first coupler 810.

[0106] The functions of the dual-mode beam splitter 600, linear operation module 100, second delay module 500, first delay module 200, nonlinear operation module 700, and filtering module 400 can be found in the foregoing description, and will not be repeated here.

[0107] Couplers are primarily used to combine optical signals output from different modules or devices. For example, the first coupler 810 can couple optical signals input to the optical computing device (such as the third group of optical signals) and optical signals output from the filtering module 400 (the second group of optical signals, the eighth group of optical signals, or the eleventh group of optical signals). The second coupler 820 is used to combine the optical signals output from the linear operation module 100 (the fourth group of optical signals) and the optical signals output from the second delay module 500 (the second group of optical signals after time delay adjustment). The third coupler 830 can combine the optical signals output from the second coupler 820 and the optical signals output from the nonlinear operation module 700.

[0108] Optical switches can change the transmission direction of optical signals. For example, the first optical switch 910 can transmit the signal output by the first delay module 200 to the filter module 400 or the nonlinear operation module 700. The second optical switch 920 can transmit the optical signal output by the nonlinear operation module 700 to the third coupler 830 or output the optical signal output by the nonlinear operation module 700.

[0109] The following is combined with Figure 7 This paper describes the transmission process of optical signals within an optical computing device during computation. The optical computing device can perform two types of computations: linear and nonlinear. The transmission process of optical signals in these two types of computations is described below:

[0110] 1. The operation process of linear operations.

[0111] Taking the linear operation module 100 performing linear operations on the first group of optical signals and the third group of optical signals as an example, Figure 8a The solid black line in the image indicates the transmission path of the first set of optical signals, which is transformed into the second set of optical signals by the linear processing module 100. Figure 8a It can be seen that the seventh group of optical signals enters the optical computing device through the first coupler 810, and then passes through the dual-mode beam splitter 600. The dual-mode beam splitter 600 divides the twelfth group of optical signals into the first group of optical signals and the third group of optical signals. The first group of optical signals is transmitted to the linear operation module 100. The linear operation module 100 modulates the first group of optical signals according to the received first electrical signal and outputs the second group of optical signals. The second group of optical signals is transmitted through the second coupler 820 to the third coupler 830. The third coupler 830 transmits the second group of optical signals to the first delay module 200. The first delay module 200 adjusts the time delay of the second group of optical signals. The second group of optical signals with the time delay adjusted is transmitted to the filtering module 400 under the action of the first optical switch 910. The filtering module 400 can transmit the second group of optical signals with the time delay adjusted directly to the first coupler 810 without performing any rejection operation.

[0112] The third set of optical signals is transmitted in the optical computing device without passing through the linear operation module 100. The transmission path is: second delay module 500 -> second coupler 820 -> third coupler 830 -> first delay module 200 -> first optical switch 910 -> filter module 400 -> first coupler 810 -> dual-mode beam splitter 600. After the last optical signal in the first set of optical signals is input to the linear operation module 100, the dual-mode beam splitter 600 transmits the third set of optical signals to the linear operation module 100.

[0113] Figure 8a The black dashed line marks the transmission path of the fourth optical signal after the third optical signal is converted into the fourth optical signal by the linear processing module 100. Figure 8a It can be seen that the dual-mode beam splitter 600 transmits the third set of optical signals to the linear operation module 100. The third set of optical signals is input to the linear operation module 100. The linear operation module 100 modulates the third set of optical signals according to the received second electrical signal and outputs the fourth set of optical signals.

[0114] Figure 8a The transmission path of the second set of optical signals after passing through the first coupler 810 is marked with a solid gray line. The second set of optical signals passes through the dual-mode beam splitter 600, which transmits the second set of optical signals to the second delay module 500. The second delay module 500 further adjusts the time delay of the second set of optical signals so that after the second set of optical signals is output from the second delay module 500, it can arrive at the second coupler 820 at the same time as the fourth set of optical signals.

[0115] The second coupler 820 couples the second group of optical signals and the fourth group of optical signals, outputs the fifth group of optical signals, and transmits the fifth group of optical signals to the third coupler 830.

[0116] If the optical computing device no longer needs to receive the optical signal indicating the first set of data, that is, the calculation performed by the optical computing device has been completed, the fifth set of optical signals output by the third coupler 830 can be output from the optical computing device through the first delay module 200 (the first delay module 200 does not perform any operation), the first optical switch 910, the nonlinear operation module 700 (the nonlinear operation module 700 does not perform any operation), and the second optical switch 920. The fifth set of optical signals can indicate the final calculation result.

[0117] If the optical computing device needs to continue receiving optical signals indicating the first set of data, the third coupler 830 can continue to transmit the fifth set of optical signals to the first delay module 200. The transmission process of the fifth set of optical signals is similar to the transmission process of the second set of optical signals after being output from the third coupler 830, as detailed above. The third coupler 830 can combine the delayed fifth set of optical signals and other sets of optical signals modulated by the linear operation module 100 to output a sixth set of optical signals. The sixth set of optical signals output by the third coupler 830 can be output from the optical computing device through the first delay module 200 (the first delay module 200 does not perform any operation), the first optical switch 910, the nonlinear operation module 700 (the nonlinear operation module 700 does not perform any operation), and the second optical switch 920. The fifth set of optical signals can indicate the final calculation result.

[0118] Figure 8a The optical signal transmission process shown only involves linear operations. When the optical computing device needs to perform pooling operations, the optical signal coupled by the third coupler 830 after the linear operations still needs to be transmitted in the optical computing device for further processing.

[0119] Taking the sixth group of optical signals requiring pooling as an example, for example... Figure 7The process of optical signal transmission in the optical computing device during pooling operation is explained. The fifth optical signal output from the third coupler 830 can pass through the first delay module 200 (the first delay module 200 does not perform any operation), the first optical switch 910, the filtering module 400 (the filtering module 400 does not perform any operation), the first coupler 810, and the dual-mode beam splitter 600, and finally reach the linear operation module 100. The linear operation module 100 modulates the sixth electrical signal according to the received third electrical signal indicating the second weight and outputs the seventh optical signal. The seventh optical signal passes through the second coupler 820, the third coupler 830, the first delay module 200 (the first delay module 200 does not perform any operation), and the first optical switch 910, and reaches the filtering module 400. The filtering module 400 filters out invalid signals in the seventh optical signal and outputs the eighth optical signal. The eighth optical signal passes through the second coupler 820, the third coupler 830, the first delay module 200, the first optical switch 910, the nonlinear operation module 700 (the nonlinear operation module 700 does not perform any operation), and the second optical switch 920, and is output from the optical computing device.

[0120] Taking the ninth group of optical signals that needs to undergo convolution operations as an example, for example... Figure 7 The transmission process of optical signals in the optical computing device during convolution operations is explained below. The linear operation module 100 receives the ninth group of optical signals. The linear operation module 100 modulates the ninth group of optical signals according to the received fourth electrical signal, and outputs the tenth group of optical signals. (Here, it is only an example where the linear operation module 100 outputs the tenth group of optical signals indicating the result of the multiplication and addition operation of the second group of data and the second weight, and only modulates it once. In actual operation, the generation of the tenth group of optical signals can be similar to the generation process of the fifth or sixth group of optical signals. That is, the linear operation module 100, the first delay module 200, the second delay module 500, the second coupler 820, and the third coupler 830 can cooperate to generate the tenth group of optical signals.) The optical signal is transmitted to the third coupler 830 via the second coupler 820. The tenth group of optical signals passes through the third coupler 830, the first delay module 200 (the first delay module 200 does not perform any operation), and the first optical switch 910, and reaches the filtering module 400. The filtering module 400 filters out invalid signals in the tenth group of optical signals and outputs the eleventh group of optical signals. The eleventh group of optical signals passes through the second coupler 820, the third coupler 830, the first delay module 200, the first optical switch 910, the nonlinear operation module 700 (the nonlinear operation module 700 does not perform any operation), and the second optical switch 920, and is output from the optical computing device.

[0121] 2. The calculation process of nonlinear operations.

[0122] The following explanation will continue using the example of an optical computing device performing nonlinear operations on the sixth group of optical signals.

[0123] Figure 8b The transmission path of the fifth group of optical signals is marked with a solid black line. If the subsequent optical computing device does not need to continue receiving the optical signals indicating the first group of data, that is, the linear operations required by the optical computing device have been completed, the third coupler 830 transmits the fifth group of optical signals to the first delay module 200. The first delay module 200 may not process the fifth group of optical signals and transmits the fifth group of optical signals to the first optical switch 910. Through the first optical switch 910, the fifth group of optical signals can be transmitted to the nonlinear operation module 700. The nonlinear operation module 700 can perform nonlinear operations on the fifth group of optical signals and then transmit the fifth group of optical signals after the nonlinear operation to the second optical switch 920. Through the second optical switch 920, the fifth group of optical signals after the nonlinear operation is output from the optical computing device, and the fifth group of optical signals after the nonlinear operation can indicate the final calculation result.

[0124] Figure 8b The black dashed line marks an alternative transmission path for the fifth group of optical signals. If the optical computing device needs to continue processing the fifth group of optical signals, the third coupler 830 can continue to transmit the fifth group of optical signals to the first delay module 200. The transmission process of the fifth group of optical signals is similar to the transmission process of the second group of optical signals after being output from the third coupler 830, as detailed above, and will not be repeated here. Until all the linear operations required by the linear operation module 100 are completed, the third coupler 830 can receive the merged optical signals from the second coupler 820. Here, the optical signal received by the third coupler 830 from the second coupler 820 after the linear operation module 100 has performed all linear operations is considered the sixth group of optical signals. Figure 8b The transmission path of the sixth group of optical signals is marked with a gray dashed line. The second coupler 820 receives the sixth group of optical signals and transmits it to the first delay module 200. The first delay module 200 may not process the sixth group of optical signals and transmits it to the first optical switch 910. Through the first optical switch 910, the sixth group of optical signals can be transmitted to the nonlinear operation module 700. The nonlinear operation module 700 can perform nonlinear operations on the sixth group of optical signals and then transmit the sixth group of optical signals after the nonlinear operation to the second optical switch 920. Through the second optical switch 920, the sixth group of optical signals after the nonlinear operation is output from the optical computing device, indicating the final calculation result.

[0125] It should be noted that, Figures 8a-8b Amplifier 1000 is not shown in the figure. Amplifier 1000 can amplify the optical signal output by the third coupler 830.

[0126] The following describes the composition of each module or device in the optical computing device and the process of processing optical signals:

[0127] (1) Linear operation module 100.

[0128] The linear operation module 100 can modulate the received optical signals (first group of optical signals, third group of optical signals and sixth group of optical signals) according to the received electrical signals (first electrical signal, second electrical signal or third electrical signal), change the light intensity of the optical signals, and realize linear operation.

[0129] In order to achieve linear operation, after receiving the optical signal, the linear operation module 100 can divide the optical signal into N optical signals with different time delays, and then modulate the N optical signals with different time delays.

[0130] The linear operation module 100 includes multiple beam splitters, multiple delay waveguides, multiple couplers, and N optoelectronic modulators. Optionally, it may also include a delay control module.

[0131] The delay control module receives the optical signal input to the linear operation module and calibrates the delay of the optical signal. The function of the delay control module, the first delay module 200, and the second delay module 500 is to adjust the delay of the optical signal. The structures of the delay control module, the first delay module 200, and the second delay module 500 are also similar. For details on the structure of the delay control module, please refer to the descriptions of the first delay module 200 and the second delay module 500 below; they will not be repeated here.

[0132] A beam splitter can divide a single optical signal into two identical optical signals. The beam splitter can be a Y-shaped beam splitter or a multimode interference coupler. Any method capable of dividing a single optical signal into two identical optical signals is applicable to the embodiments of this application. A time-delay waveguide can transmit optical signals and also adjust the time delay of the transmitted optical signals.

[0133] Multiple beam splitters and multiple delay waveguides work together to divide the optical signal output by the delay control module into N optical signals with different time delays.

[0134] An optoelectronic modulator is used to modulate an optical signal based on a received electrical signal. The optoelectronic modulator can be an MZI (micro-zinc ring inductor) or a tunable micro-ring resonator. Each optoelectronic modulator modulates one of N different optical signals output from multiple beamsplitters and multiple delay waveguides.

[0135] Multiple couplers work together to combine the optical signals output from N optoelectronic modulators into a single optical signal. The coupler can be a Y-beam splitter, achieving its function through a reverse Y-beam splitter.

[0136] like Figure 9a As shown, taking N=4 as an example, this is a schematic diagram of the structure of the linear operation module 100. The linear operation module 100 includes a time delay control module 1100, three beam splitters (distinguished by beam splitter 1101, beam splitter 1102, and beam splitter 1103 respectively), three delay waveguides (distinguished by delay waveguide 1104, delay waveguide 1105, and delay waveguide 1106 respectively), four optoelectronic modulators (distinguished by optoelectronic modulator 1107, optoelectronic modulator 1108, optoelectronic modulator 1109, and optoelectronic modulator 1110 respectively), and three couplers (distinguished by coupler 1111, coupler 1112, and coupler 1113 respectively).

[0137] For example Figure 9a Taking the linear operation module 100 receiving the first group of optical signals as an example, the first group of optical signals is input to the delay control module 1100, and the delay control module 1000 adjusts the delay of the first group of optical signals.

[0138] The first group of optical signals, after time delay adjustment, is transmitted to beam splitter 1101 and split into two identical first group optical signals. For ease of explanation, these two first group optical signals are referred to as first group optical signal 1 and second group optical signal 2. First optical signal 1 does not need to pass through the delay waveguide and is transmitted to beam splitter 1102, where it is split into two identical first group optical signals. For ease of explanation, these two first group optical signals are referred to as first group optical signal 3 and second group optical signal 4. First group optical signal 3 is transmitted to opto-modulator 1107. Second group optical signal 4 passes through delay waveguide 1105, which adjusts the time delay of second group optical signal 4. The time-delayed first group optical signal 4 is then transmitted to opto-modulator 1108.

[0139] The first optical signal 2 passes through the delay waveguide 1104 and is transmitted to the beam splitter 1103, where it is split into two identical first group optical signals. For ease of explanation, these two first group optical signals are referred to as the first group optical signal 5 and the second group optical signal 6. The first group optical signal 5 is transmitted to the optoelectronic modulator 1109. The second group optical signal 6 passes through the delay waveguide 1106, which adjusts the time delay of the second group optical signal 6. The first group optical signal 4 with the adjusted time delay is then transmitted to the optoelectronic modulator 1110.

[0140] The first electrical signal received by the linear operation module 100 can be loaded onto four photoelectric modulators so that the four photoelectric modulators can modulate the received first set of optical signals based on the first electrical signal.

[0141] It should be noted that the first electrical signal may include multiple sub-signals, with one sub-signal loaded on each photoelectric modulator, and each photoelectric modulator modulates the received first set of optical signals according to the loaded electrical signal.

[0142] The linear operation module 100 may further include N phase modulators, each phase modulator being connected to a photoelectric modulator, and the phase modulators being able to modulate the phase of the optical signal output by the photoelectric modulators. The linear operation module 100 may further include at least N-1 phase-locked loop (PLL) modules, which are used to stabilize the phase of the optical signal.

[0143] See Figure 9b When N equals 4, the linear operation module provided in this application embodiment is similar to... Figure 9a Unlike other linear operation modules, the linear operation module 100 includes four phase modulators, namely phase modulator 1114, phase modulator 1115, phase modulator 1116 and phase modulator 1117, and also includes three phase-locked units, namely phase-locked unit 1118, phase-locked unit 1119 and phase-locked unit 1120.

[0144] Phase modulator 1114 is connected to photoelectric modulator 1107 to adjust the phase of the optical signal output by photoelectric modulator 1107. Phase modulator 1115 is connected to photoelectric modulator 1108 to adjust the phase of the optical signal output by photoelectric modulator 1108. Phase modulator 1116 is connected to photoelectric modulator 1109 to adjust the phase of the optical signal output by photoelectric modulator 1109. Phase modulator 1117 is connected to photoelectric modulator 1110 to adjust the phase of the optical signal output by photoelectric modulator 1107.

[0145] Phase-locked unit 1118 is connected to phase modulator 1114 and locks the phase of the optical signal output by phase modulator 1114 according to the optical signal output by coupler 1111. Phase-locked unit 1119 is connected to phase modulator 1116 and locks the phase of the optical signal output by phase modulator 1116 according to the optical signal output by coupler 1112. Phase-locked unit 1120 is connected to coupler 1112 and locks the phase of the optical signal output by coupler 1112 according to the optical signal output by coupler 1113.

[0146] (2) First delay module 200, second delay module 500.

[0147] In the optical computing device, the function of the first delay module 200 and the second delay module 500 is to adjust the delay of the optical signal. The embodiments of this application do not limit the structure of the first delay module 200 and the second delay module 500. Any device that can adjust the delay of the optical signal is applicable to the embodiments of this application.

[0148] The structure of a first delay module 200 provided in an embodiment of this application is described below. See also... Figure 10 The first delay module 200 can adopt a cascaded structure, and the first delay module 200 can include multiple optical switches connected in series (in Figure 10 The optical switches are distinguished by optical switches 1, 2, ..., n. Two optical signal transmission paths are established between the two optical switches: a long-delay transmission path with an adjustable delay device (such as delay waveguide 1) that adds a longer delay to the optical signal, and a short-delay transmission path with an adjustable delay device (such as delay waveguide 2) that adds a shorter delay to the optical signal. The delay added by the short-delay device is shorter than that added by the adjustable delay device on the long-delay transmission path. Alternatively, the short-delay transmission path may not have an adjustable delay device. In the first delay module 200, each optical switch can be controlled by an electrical signal to select the optical signal transmission path.

[0149] (3) Filtering module 400.

[0150] The filtering module 400 can remove some optical signals from the serial optical signal and adjust the time delay of the optical signal transmitted before the removed optical signal to ensure that there are no gaps in the set of optical signals output by the filtering module 400.

[0151] like Figure 11 The diagram shown is a structural schematic of a filtering module provided in an embodiment of this application. The filtering module 400 includes two optical switches (distinguished by optical switch 501 and optical switch 502 for ease of explanation) and a third delay module 503.

[0152] Taking the optical signal received by the filter module 400 as the fifth group of optical signals as an example, the transmission process of the optical signal in the filter module 400 is explained below:

[0153] After the seventh group of optical signals is input to the filtering module 400, it first enters the optical switch 501. Under the action of the optical switch 501, it is transmitted to the third delay module 503. The delay module 503 can directly transmit the seventh group of optical signals to the optical switch 502 without processing. The optical switch 502 outputs the invalid signals in the seventh group of optical signals to the outside of the optical computing device and transmits the remaining optical signals in the seventh group of optical signals (the eighth group of optical signals) to the optical switch 501. The optical switch 501 transmits the eighth group of optical signals to the third delay module 503. Since the signals included in the seventh group of optical signals are serial optical signals, when a part of the optical signals are removed, there will be a gap in the eighth group of optical signals. In order to reduce this gap, the third delay module 503 can adjust the delay of the optical signals in the eighth group that are transmitted before the invalid signals, but does not adjust the delay of the optical signals that are transmitted before the invalid signals, so as to ensure that there is no gap between the optical signals that are transmitted before the invalid signals and the optical signals that are transmitted after the invalid signals.

[0154] (4) Dual-mode beam splitter 600, optical switches (first optical switch 910 and second optical switch 920).

[0155] The dual-mode beam splitter 600 and optical switch are essentially beam splitters with beam splitting ratio adjustment functions.

[0156] Taking the dual-mode beam splitter 600 as an example, the structure of the dual-mode beam splitter 600 and the optical switch is described as follows: Figure 12 As shown, the dual-mode beam splitter 600 includes a beam splitter 601, two phase shifters (referred to as phase shifter 602 and phase shifter 603 for ease of explanation) and a coupler 604. The beam splitter 601 can split an optical signal into two identical optical signals, and the phase shifters 602 and 603 can compensate for the phase of the optical signals, thereby adjusting the beam splitting ratio.

[0157] (5) Couplers (such as coupler 300, first coupler 810 and second coupler 820, and coupler 604).

[0158] In this embodiment, the coupler can be a Y-type coupler, used to couple two optical signals into one optical signal.

[0159] (6) Nonlinear operation module 700.

[0160] The embodiments of this application do not limit the composition of the nonlinear operation module 700. For example, the nonlinear operation module 700 may include devices with optical nonlinear effects, such as graphene layers.

[0161] Combination Figure 7 The optical computing device shown is described, and the method by which the optical computing device implements the operation process of fully connected layers, convolutional layers and pooling layers is explained.

[0162] 1. Fully connected layer.

[0163] To realize the operation of the fully connected layer, the optical computing device can be used to implement the multiplication and addition operation. The optical computing device can receive serial optical signals multiple times (such as the first group of optical signals or the third group of optical signals). For ease of explanation, Xc[1]~Xc[n] is used to represent the serial optical signal. The serial optical signal is composed of n sub-signals (such as Xc[1], Xc[n]).

[0164] After the serial optical signals Xc[1]~Xc[n] enter the optical computing device, they are transmitted sequentially to the linear operation module 100. In the linear operation module 100, for any sub-signal, the linear operation module 100 can modulate the sub-signal according to the received electrical signal (such as the first electrical signal and the second electrical signal). The modulated sub-signal continues to be transmitted in the optical computing device. The modulated sub-signal passes through the first delay module 200 (adjusts the time delay of the modulated sub-signal so that the adjusted sub-signal can be aligned with the modulated sub-signals in the serial optical signals Xc[1]~Xc[n] input to the optical computing device in the next time in terms of time delay), the filtering module 400 (the filtering module 400 does not perform any processing), the dual-mode beam splitter 600, the second delay module 500, the second coupler 820, and the third coupler 830.

[0165] The optical computing device can continue to receive serial optical signals Xc[1]~Xc[n] to perform the above operations until all linear operations are completed.

[0166] The following example, using n=8 and the linear operation module 100 including 4 adjusters, illustrates the operation process performed by the optical computing device:

[0167] See Figure 13 During time periods 1 to 8, the linear operation module 100 sequentially receives Xc[1] to Xc[8]. The four modulators in the linear operation module 100 can adjust the intensity of Xc[1] to Xc[8]. ij It can indicate the degree of modulation of the sub-signal light intensity by the modulator in the linear operation module 100 (that is, w ij (can indicate weights), where i indicates the number of the sub-signal adjusted by the modulator, and j indicates the number of the opto-modulator.

[0168] Taking Xc[1] as an example, after Xc[1] is input to the linear operation module 100 in time period 1, it is split into four sub-signals Xc[1] with different delays, namely Xc[1] with a delay in time period 1, Xc[1] with a delay in time period 2, Xc[1] with a delay in time period 3, and Xc[1] with a delay in time period 4. The sub-signal modulated by the first photoelectric modulator after Xc[1] with a delay in time period 1 is w 11 Xc[1]. The second photoelectric modulator modulates the sub-signal w of Xc[1] with a time delay of 2. 12 Xc[1]. The third photoelectric modulator modulates the sub-signal w of Xc[1] with a time delay of 3. 13 Xc[1]. The fourth photoelectric modulator modulates the sub-signal w of Xc[1] with a time delay of 4. 14 Xc[1].

[0169] The modulated sub-signal continues to be transmitted in the optical computing device. The transmission path is: second coupler 820 -> third coupler 830 -> first delay module 200 -> filter module 400 -> dual-mode beam splitter 600 -> second delay module 500, until the last sub-signal in the serial optical signal passes through the linear operation module 100.

[0170] Afterwards, the first delay module 200 adjusts the delay of all the modulated sub-signals, and the increased delay is 2*4θ (θ is the time length of a single signal pulse). At this time, the optical computing device continues to receive serial optical signals Xc[1]~Xc[8]. The subsequent serial optical signals Xc[1]~Xc[8] are processed in the optical computing device in a similar manner to the previous serial optical signals Xc[1]~Xc[8].

[0171] After the last sub-signal in the serial optical signal passes through the linear operation module 100, it passes through the second coupler 820 and the third coupler 830, and the optical signal is merged into 8 serial optical signals, which can indicate the final calculation result.

[0172] 2. Convolutional layer.

[0173] The calculation process of convolutional layers is similar to that of fully connected layers. The difference is that when performing the operation of convolutional layers, the filtering module 400 can remove some optical signals that do not need to participate in the operation from the optical signal modulated by the linear operation module 100. It should be noted that the optical signals removed by the filtering module 400 are related to the operation to be performed.

[0174] Taking the convolution operation of a 3x3 matrix and a 2x2 kernel as an example, the convolution operation of a 3x3 matrix and a 2x2 kernel is as follows: Figure 14As shown, Xc1 to Xc9 are the serial optical signals to be input, and a, b, c, and d are the weights indicated by the convolution kernel. Xn[1] to Xn[4] are the optical signals to be output, indicating the final calculation result.

[0175] The following example, using n=9 and two adjusters in the linear operation module 100 participating in the calculation, illustrates the convolution operation performed by the optical computing device:

[0176] See Figures 15a-15d The linear operation module 100 receives Xc1 to Xc9 sequentially. The two modulators in the linear operation module 100 can adjust the intensity of Xc[1] to Xc[9]. d and c respectively indicate the degree of modulation of the sub-signal light intensity by the modulator during the first linear operation of the linear operation module 100 (i.e., the weight indicated by the convolution kernel). a and b respectively indicate the degree of modulation of the sub-signal light intensity by the modulator during the second linear operation of the linear operation module 100 (i.e., the weight indicated by the convolution kernel). Xn1 to Xn9 are the optical signals output by the linear operation module 100 during the first linear operation.

[0177] Figure 15a shows the input serial optical signal and the output optical signal during the first linear operation of the linear operation module 100. Figure 15b shows the input serial optical signal and the output optical signal during the second linear operation of the linear operation module 100. Figure 15c shows the optical signals removed by the filtering module 400 from the output optical signal of the linear operation module 100 (the optical signals in the linear box are the removed signals Xn1~Xn4, Xn7) and the remaining optical signal. Figure 15d shows the final output optical signal of the optical computing device.

[0178] 3. Pooling layer.

[0179] The operation process of pooling layers is similar to that of convolutional layers. The difference is that the function of convolutional kernels in pooling layers is to retain the maximum value (max pooling) or the average value (average pooling) of specific data.

[0180] The following example illustrates the average pooling operation performed by an optical computing device to convert a 4x4 matrix into a 2x2 matrix:

[0181] Average pooling operation from 4x4 matrix to 2x2 matrix is ​​as follows Figure 16 As shown, average pooling can be used to reduce the dimensionality of data.

[0182] See Figure 17 , Figure 17 The linear operation module 100 receives the serial optical signal as input and the optical signal as output during the linear operation process. The linear operation module 100 sequentially receives Xc1 to Xc... 16Wherein, 0.25 indicates the degree of modulation of the sub-signal light intensity by the regulator during the linear operation of the linear operation module 100, and the blank space indicates that the regulator modulates the sub-signal light intensity to 0 during the linear operation. Xn1~Xn 16 The optical signal output by the linear operation module 100. Figure 17 As can be seen, the valid signals are Xn6, Xn8, Xn14, and Xn16. Xn1 to Xn5, Xn7, Xn9 to Xn13, and Xn15 are all invalid signals. The filtering module 400 in the optical computing device can filter out the invalid signals.

[0183] To make the solution clearer, the following will combine the previous embodiments with... Figure 18 Taking the calculation method shown as an example, this paper provides a general overview of the optical calculation method performed by the optical computing device provided in the embodiments of the present invention.

[0184] The linear operation module 100 receives a first set of optical signals and a first electrical signal. The linear operation module 100 modulates the first set of optical signals according to the first electrical signal to output a second set of optical signals.

[0185] The linear operation module 100 can receive a third set of optical signals and a second electrical signal. The linear operation module 100 modulates the third set of optical signals according to the second electrical signal to output a fourth set of optical signals. The fourth set of optical signals is used to indicate the calculation result of the first set of data and the second part of the weight value.

[0186] The first delay module 200 can adjust the delay of the second set of optical signals; then, the coupler 300 can merge the delayed second set of optical signals with the fourth set of optical signals into a fifth set of optical signals, wherein the fifth set of optical signals indicates the first calculation result after multiplying and adding the first set of data with the first part of the weight and the second part of the weight.

[0187] In one possible implementation, the optical computing device may further include a filtering module, as illustrated by the example of the optical computing device performing a pooling operation on the sixth group of optical signals. The linear operation module 100 can receive the sixth group of optical signals and the third electrical signal.

[0188] The linear operation module 100 modulates the sixth group of optical signals according to the third electrical signal to output the seventh group of optical signals. The filtering module can receive the seventh group of optical signals, filter out invalid signals in the seventh group of optical signals, and output the eighth group of optical signals.

[0189] In one possible implementation, where the optical computing device includes a filtering module, the optical computing device can also perform convolution operations using the filtering module. The following explanation uses the example of the optical computing device performing a convolution operation on the ninth group of optical signals.

[0190] The linear operation module 100 can receive the ninth group of optical signals and the fourth electrical signal, and modulate the ninth group of optical signals according to the fourth electrical signal to output the tenth group of optical signals.

[0191] The filtering module receives the tenth group of optical signals, filters out invalid signals in the tenth group of optical signals, and outputs the eleventh group of optical signals. The eleventh group of optical signals is used to indicate the calculation result of the convolution operation on the third calculation result.

[0192] This application also provides an optical computing system, which includes a processor and the optical computing device mentioned above. See [link to previous document]. Figure 19 The optical computing system 10 includes an optical computing device 20 and a processor 30. The processor 30 and the optical computing device 20 can be connected via a standard host interface or network interface. For example, the host interface may include a Peripheral Component Interconnect Express (PCIE) interface. The processor 30 can send data to be computed to the optical computing device 20, wherein the data includes a first set of data and may also include a first weight. The data may also be a second set of data and a second weight. When the optical computing device 20 receives the data, it can determine one or more sets of optical signals (such as a first set of optical signals, a third set of optical signals, a ninth set of optical signals, etc.) and electrical signals (a first electrical signal, a second electrical signal, and a fourth electrical signal, etc.) based on the data to complete the corresponding computation process.

[0193] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0194] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0195] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0196] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An optical computing device, characterized in that, include: A linear operation module is used to receive a first set of optical signals and a third set of optical signals respectively, wherein the first set of optical signals and the third set of optical signals are used to indicate the first set of data; The system receives a first electrical signal indicating a first portion of the weight value in the first weight, and modulates the first set of optical signals according to the first electrical signal to output a second set of optical signals, wherein the second set of optical signals is used to indicate the calculation result of the first set of data and the first portion of the weight value, and the first weight is the weight of the first neural network layer. The system receives a second electrical signal indicating the second portion of the weight in the first weight, and modulates the third set of optical signals according to the second electrical signal to output a fourth set of optical signals, wherein the fourth set of optical signals is used to indicate the calculation result of the first set of data and the second portion of the weight value; The first delay module is used to adjust the time delay of the second group of optical signals; A coupler is used to combine the delayed second group of optical signals with the fourth group of optical signals into a fifth group of optical signals, wherein the fifth group of optical signals indicates the first calculation result after the first group of data is multiplied and added with the first part of the weights and the second part of the weights.

2. The optical computing device as described in claim 1, characterized in that: The linear operation module is further configured to receive a sixth set of optical signals, the sixth set of optical signals being used to indicate a second calculation result after the first set of data and the first weight are multiplied and added, wherein the second calculation result includes the first calculation result; The system receives a third electrical signal indicating a second weight and modulates the sixth group of optical signals according to the third electrical signal to output a seventh group of optical signals, wherein the second weight is the weight of the second neural network layer. The optical computing device further includes: The filtering module is used to receive the seventh set of optical signals, filter out invalid signals in the seventh set of optical signals, and output an eighth set of optical signals. The eighth set of optical signals is used to indicate the calculation result of the pooling operation on the second calculation result.

3. The optical computing device as described in claim 2, characterized in that: The linear operation module is also used to receive a ninth set of optical signals, which are used to indicate the second set of data; The system receives a fourth electrical signal indicating a third weight and modulates the ninth group of optical signals according to the fourth electrical signal to output a tenth group of optical signals. The tenth group of optical signals is used to indicate the third calculation result after the second group of data is multiplied and added with the third weight. The third weight is the weight of the third neural network layer. The filtering module in the device is also used to receive the tenth group of optical signals, filter out invalid signals in the tenth group of optical signals, and output the eleventh group of optical signals. The eleventh group of optical signals is used to indicate the calculation result of the convolution operation on the third calculation result.

4. The optical computing device as described in claim 3, characterized in that, The device further includes: The nonlinear operation module is used to perform nonlinear operations on the sixth group of optical signals, the fifth group of optical signals, the eighth group of optical signals, or the eleventh group of optical signals, wherein the nonlinear operation is an activation function operation.

5. The optical computing device as described in any one of claims 1, 2, and 4, characterized in that, The device further includes: Dual-mode beam splitter, Used to receive a twelfth group of optical signals used to indicate the first group of data, and to divide the twelfth group of optical signals into the first group of optical signals and the third group of optical signals.

6. The optical computing device as described in any one of claims 1, 2, and 4, characterized in that, The device also includes a second delay module. The dual-mode beam splitter in the device is also used to transmit the delayed second set of optical signals to the second delay module; The second delay module is also used to adjust the time delay of the second group of optical signals so that the second group of optical signals is transmitted to the coupler simultaneously with the fourth group of optical signals.

7. The optical computing device as described in any one of claims 1, 2, and 4, characterized in that, The linear operation module includes N photoelectric modulators. Each of the N photoelectric modulators is configured to receive one optical signal from the first group of optical signals and one electrical signal from the first electrical signal indicating the first part weight, and modulate the received optical signal according to the received electrical signal.

8. The optical computing device as described in claim 7, characterized in that, The linear operation module also includes N phase modulators and at least N-1 phase-locked loop modules; One of the phase modulators is connected to one of the photoelectric modulators for adjusting the phase of a first group of optical signals or a second group of optical signals modulated by the modulator; One of the phase-locked modules is connected to two of the N phase modulators and is used to lock the phase of the first or second set of optical signals after being modulated by the two phase modulators.

9. The optical computing device as described in claim 3, characterized in that, The linear operation module also includes: An amplifier is used to amplify the power of the fifth group of optical signals, the sixth group of optical signals, the eighth group of optical signals, or the eleventh group of optical signals.

10. A calculation method, characterized in that, Applied to an optical computing device, the optical computing device including a linear operation module, a first delay module, and a coupler, the method includes: The linear operation module receives a first set of optical signals and a first electrical signal, wherein the first set of optical signals is used to indicate a first set of data, and the first electrical signal is used to indicate a first part of the weight value in the first weight; the first set of optical signals is modulated according to the first electrical signal to output a second set of optical signals, wherein the second set of optical signals is used to indicate the calculation result of the first set of data and the first part of the weight value, and the first weight is the weight of the first neural network layer; The linear operation module receives a third set of optical signals and a second set of electrical signals. The third set of optical signals is used to indicate the first set of data, and the second set of electrical signals is used to indicate the second part of the weight value in the first weight. The module modulates the third set of optical signals according to the second set of electrical signals to output a fourth set of optical signals. The fourth set of optical signals is used to indicate the calculation result of the first set of data and the second part of the weight value. The first delay module adjusts the delay of the second group of optical signals; The coupler combines the delayed second group of optical signals with the fourth group of optical signals into a fifth group of optical signals, wherein the fifth group of optical signals indicates the first calculation result after the first group of data is multiplied and added with the first part of the weights and the second part of the weights.

11. The calculation method as described in claim 10, characterized in that, The optical computing device further includes a filtering module, and the method further includes: The linear operation module receives a sixth set of optical signals, which are used to indicate the second calculation result after the first set of data and the first weight are multiplied and added, and a third electrical signal used to indicate the second weight. The second calculation result includes the first calculation result. The second weight is the weight of the second neural network layer. The linear operation module modulates the sixth group of optical signals according to the third electrical signal to output the seventh group of optical signals; The filtering module receives the seventh set of optical signals, filters out invalid signals in the seventh set of optical signals, and outputs an eighth set of optical signals. The eighth set of optical signals is used to indicate the calculation result of pooling the second calculation result.

12. The calculation method as described in claim 10 or 11, characterized in that, Also includes: The linear operation module receives the ninth set of optical signals, which are used to indicate the second set of data. The linear operation module receives a fourth electrical signal indicating the third weight, and modulates the ninth group of optical signals according to the fourth electrical signal to output a tenth group of optical signals. The tenth group of optical signals is used to indicate the third calculation result after the second group of data is multiplied and added with the third weight. The third weight is the weight of the third neural network layer. The filtering module in the device receives the tenth set of optical signals, filters out invalid signals in the tenth set of optical signals, and outputs the eleventh set of optical signals. The eleventh set of optical signals is used to indicate the calculation result of the convolution operation on the third calculation result.

13. An optical computing system, characterized in that, The device includes a processor and an optical computing apparatus as described in any one of claims 1-9 connected to the processor, wherein the processor is configured to send data to be computed to the optical computing apparatus, wherein the data includes the first set of data.

Images