Model Training Method, Device, Storage Medium, and Program Product

Abstract

The present application discloses a model training method, device, storage medium, and program product, relating to the field of artificial intelligence computing technology. When receiving a model training task transmitted by an electrical signal, it is possible to extract a matrix multiplication operation from the model training task, and set the transformation parameters of a photonic arithmetic unit according to the electrical signal corresponding to the second matrix in the matrix multiplication operation. In this way, after converting the electrical signal corresponding to the first matrix in the matrix multiplication operation into a first optical signal, and inputting the first optical signal into the photonic arithmetic unit, the matrix multiplication operation can be completed. Utilizing the parallel propagation characteristic of light, the photonic arithmetic unit can complete the matrix multiplication operation at the speed of light, with the delay only determined by the optical path length (nanosecond level) and without processing each data bit one by one. Therefore, the operation efficiency can be greatly improved, thereby solving the problem of low model training efficiency in the related art.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence computing technology, and particularly to a model training method, device, storage medium, and program product. Background Art

[0002] Model training is a key link in the learning of artificial intelligence algorithms. During some model training processes, a large amount of pre-annotated training data needs to be input into the model. Based on the training data, the model can output prediction results. Based on the difference between the true annotation and the prediction result, the model parameters can be adjusted, so as to improve the prediction accuracy of the model. Currently, model training involves a large amount of operations, and these operations are time-consuming, reducing the model training efficiency. Summary of the Invention

[0003] This application provides a model training method, a model training device, an electronic device, a computer-readable storage medium, and a computer program product to at least solve the problem of low model training efficiency in related technologies.

[0004] This application provides a model training method, the method including:

[0005] Receiving a model training task transmitted by an electrical signal, and extracting a matrix multiplication operation from the model training task, the matrix multiplication operation including a first matrix and a second matrix;

[0006] Setting transformation parameters of a photonic arithmetic unit according to the electrical signal corresponding to the second matrix, the transformation parameters representing matrix transformations to be performed on the first matrix in the matrix multiplication operation;

[0007] Converting the electrical signal corresponding to the first matrix into a first optical signal, and inputting the first optical signal into the photonic arithmetic unit to obtain an operation result of the matrix multiplication operation;

[0008] Performing model training according to the operation result of the matrix multiplication operation.

[0009] This application further provides a model training device, the model training device including a central processing unit, an optoelectronic conversion module, a graphics processing unit, and an optical integrated circuit, the graphics processing unit including an electronic arithmetic unit, the optical integrated circuit including a photonic arithmetic unit, wherein:

[0010] One of the central processing unit and the graphics processing unit is configured to receive a model training task transmitted using an electrical signal, extract a matrix multiplication operation including a first matrix and a second matrix from the model training task, input an electrical signal corresponding to the first matrix into the optoelectronic conversion module, and set transformation parameters of the photonic arithmetic unit according to the electrical signal corresponding to the second matrix, where the transformation parameters represent matrix transformations to be performed on the first matrix in the matrix multiplication operation;

[0011] The optoelectronic conversion module is configured to convert the electrical signal corresponding to the first matrix into a first optical signal and input the first optical signal into the photonic arithmetic unit;

[0012] The photonic arithmetic unit is configured to perform a transformation on the first optical signal according to the transformation parameters to obtain an operation result of the matrix multiplication operation;

[0013] One of the central processing unit and the graphics processing unit performs model training according to the operation result of the matrix multiplication operation.

[0014] The present application further provides an electronic device, including: a memory configured to store a computer program; a processor configured to implement the steps of the above model training method when executing the computer program.

[0015] The present application further provides a computer-readable storage medium storing a computer program therein, where the computer program implements the steps of the above model training method when executed by a processor.

[0016] The present application further provides a computer program product including a computer program, where the computer program implements the steps of the above model training method when executed by a processor.

[0017] In the technical solutions of some embodiments of the present application, when receiving a model training task transmitted using an electrical signal, a matrix multiplication operation can be extracted from the model training task, and transformation parameters of the photonic arithmetic unit can be set according to the electrical signal corresponding to the second matrix in the matrix multiplication operation. In this way, after converting the electrical signal corresponding to the first matrix in the matrix multiplication operation into a first optical signal and inputting the first optical signal into the photonic arithmetic unit, the matrix multiplication operation can be completed. Compared with the high latency problem of an electronic arithmetic unit performing matrix multiplication operations by relying on logic gate circuits for bit-by-bit processing, the photonic arithmetic unit can complete matrix multiplication operations at the speed of light using the parallel propagation characteristics of light. The latency is only determined by the optical path length (nanosecond level) and there is no need to process data bit by bit. Therefore, the operation efficiency can be greatly improved, and thus the problem of low model training efficiency in the related art is solved. Description of the Drawings

[0018] To more clearly illustrate the embodiments of the present application, the accompanying drawings required for use in the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0019] Figure 1 Module schematic diagram of a photoelectric conversion module provided for some embodiments of the present application;

[0020] Figure 2 Module schematic diagram of a photon arithmetic unit provided for some embodiments of the present application;

[0021] Figure 3 Flow schematic diagram of a model training method provided for some embodiments of the present application;

[0022] Figure 4 Module schematic diagram of a Mach-Zehnder interferometer provided for some embodiments of the present application;

[0023] Figure 5 Topological cascade mode of a Mach-Zehnder interferometer provided for some embodiments of the present application;

[0024] Figure 6 Topological cascade mode of a Mach-Zehnder interferometer provided for some other embodiments of the present application;

[0025] Figure 7 Internal module schematic diagram of a photon arithmetic unit provided for some embodiments of the present application;

[0026] Figure 8 Module schematic diagram of a model training device provided for some embodiments of the present application;

[0027] Figure 9 Module schematic diagram of an electronic device provided for some embodiments of the present application. Detailed implementation manners

[0028] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0029] It should be noted that in the description of this application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or elements inherent to such a process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0030] To enable those skilled in the art of this technology to better understand the solution of this application, the following further detailed description of this application will be given in conjunction with the accompanying drawings and specific embodiments.

[0031] The operations in the model training process may include matrix multiplication operations and non-matrix multiplication operations. Among them, matrix multiplication operation refers to a multiplication operation involving matrix terms, such as the multiplication operation between the weight matrix of the model and the feature vector. Non-matrix multiplication operation refers to an operation that does not involve matrix terms or a non-multiplication operation involving matrix terms. For example, addition operations, subtraction operations, division operations, statistical operations, and conditional operations involving matrix terms or non-matrix terms. Also, for example, multiplication operations between non-matrix terms. For ease of understanding, the training process of a binary classification logistic regression model under a supervised learning framework will be used as an example for illustration.

[0032] Assume that the training set D of the binary classification logistic regression model is . Among them, the data point represents the M-dimensional feature vector of the data point . The superscript T indicates transposing the feature vector, that is, transposing the M-dimensional row vector into a column vector. The value of i is an integer between 1 and N (including 1 and N), and the values of M and N are integers greater than 0. is the true label of the data point , representing the actual category to which the data point belongs. For example, can take values of 0 or 1. When = 0, it means that the data point actually belongs to category 0; when = 1, it means that the data point actually belongs to category 1.

[0033] Training the binary classification logistic regression model is to input the feature vector of the data point into the binary classification logistic regression model, and let the binary classification logistic regression model predict the category to which the data point belongs. If the prediction result is the same as the true label of the data point , it means that the prediction of the binary classification logistic regression model is accurate; if the prediction result is different from the true label of the data point If the true annotation is different, the model parameters of the binary logistic regression model can be adjusted so that the prediction result is as close as possible to the true annotation of the data point. as close as possible to the true annotation.

[0034] Specifically, during the model training process, after the feature vector of the data point is input into the binary logistic regression model, the binary logistic regression model can output the probabilities that the data point belongs to each category. Among them, the category with the highest probability can be used as the prediction result of the binary logistic regression model. For example, assume that the category probabilities output by the binary logistic regression model are as follows:

[0035] The probability of = 0 is 0.85

[0036] The probability of = 1 is 0.15

[0037] That is, the probability that the data point belongs to category 0 is 0.85, and the probability that the data point belongs to category 1 is 0.15. Since the probability of category 0 is the highest, = 0 can be used as the prediction result of the binary logistic regression model. At this time, if the true annotation of the data point is also 0, it means that the prediction result of the binary logistic regression model is accurate; if the true annotation of the data point is 1, it means that the prediction result of the binary logistic regression model is inaccurate, and the model parameters of the binary logistic regression model can be adjusted to improve the accuracy of the prediction result. Based on the above description, in the binary logistic regression model, the probabilities that the data point belongs to each category can be shown as expressions (1) and (2).

[0038] (1)

[0039] (2)

[0040] Among them, represents the probability that the data point belongs to category 1, represents the probability that the data point belongs to category 0, represents the transpose of the feature vector of the data point , represents the model parameter. During the model training process, the model parameter w can be adjusted according to the true annotation of the data point and the prediction result of the binary logistic regression model.

[0041] To obtain the optimal model parameters, an objective function as shown in Expression (3) can be constructed.

[0042] (3)

[0043] Among them, , . The process of model training is to find the optimal model parameters corresponding to the minimum of the objective function .

[0044] Since the objective function has no closed-form solution (i.e., when takes the minimum value, the optimal model parameters cannot be explicitly expressed), the gradient descent algorithm can be used to obtain the optimal model parameters . Specifically, assume that the gradient vector of the objective function is . The gradient calculation formula of the objective function can be as shown in Expression (4).

[0045] (4)

[0046] Among them, , represents the j-th element in the data point , represents the j-th element in the gradient vector .

[0047] Based on the above Expressions (1) to (4), the training process of the binary classification logistic regression model includes the following steps 1) to 5).

[0048] 1) Set the model training accuracy , step size factor and initial model parameters , and set the initial value of the iteration number t = 0.

[0049] 2) According to the above Expression (4), calculate all gradient information , and based on the gradient information , form the gradient vector . Among them, represents the j-th element in the gradient vector in the t-th iteration.

[0050] 3) If , then take the model parameters obtained in the t-th iteration as the optimal model parameters , and substitute them into the objective function in Expression (3) After obtaining the corresponding value, end the process.

[0051] 4) If , then according to the iteration rule update the model parameters and execute step 5).

[0052] 5) Increment the iteration count t by 1, and repeat the above steps 2) - 4) until the optimal model parameters are obtained .

[0053] During the above training process, the gradient needs to be calculated multiple times, and the gradient calculation can be as shown in expression (5).

[0054] (5)

[0055] where X represents the data matrix composed of the feature vectors of each data point. In the data matrix X, each row represents the feature vector of a data point, represents the transpose of the data matrix X, represents the vector composed of the difference between the predicted probability and the true label of each data point.

[0056] Also, since , before calculating expression (5), it is necessary to first calculate the inner product of the feature vector of the data point and the model parameters based on expression (6).

[0057] (6)

[0058] Substitute the calculation result of expression (5) into expression (4), and then the gradient vector can be calculated.

[0059] Based on the training process of the above binary classification logistic regression model, it can be seen that during the model training process, a large number of matrix multiplication operations are involved, such as the above expressions (5) and (6), and a large number of non - matrix multiplication operations are also involved, such as in the above expression (5).

[0060] Currently, all operations during the model training process are implemented based on electronic arithmetic units integrated in a processor. Among them, the processor may include, but is not limited to, a Graphics Processing Unit (GPU), a Central Processing Unit (CPU), etc. The electronic arithmetic unit may include an analog circuit or a digital circuit. The analog circuit can perform mathematical operations using continuously varying physical quantities (such as voltage and current). The digital circuit can perform mathematical operations using discrete digital signals (such as binary signals). This operation method based on the electronic arithmetic unit has advantages such as mature technology, low cost, high integration, and good flexibility. However, when performing some operations, the operation efficiency is relatively low, which will in turn lead to the problem of low model training efficiency. For example, when the electronic arithmetic unit performs matrix multiplication operations, it depends on the logic gate circuit for bit-by-bit processing, resulting in a high latency problem, which greatly reduces the operation efficiency and thus reduces the model training efficiency. In addition, the power consumption of the electronic arithmetic unit surges with the frequency and integration. When using the electronic arithmetic unit to perform matrix multiplication operations, the power consumption is relatively large.

[0061] In view of this, the present application provides a model training method, which can solve the problem of low model training efficiency in some technologies. The model training method can be applied to a model training device. Among them, the model training device may include, but is not limited to, a server, an inference board, etc. Modules such as a central processor, a graphics processor, a photoelectric conversion module, and a photonic arithmetic unit can be set in the model training device. For the convenience of subsequent scheme elaboration, the photoelectric conversion module and the photonic arithmetic unit will be described first below.

[0062] Referring to Figure 1 , which is a schematic diagram of the photoelectric conversion module 11 provided in some embodiments of the present application. Figure 1In [the figure], the optoelectronic conversion module 11 includes an opto-electric conversion sub-module 111 and an electro-optic conversion sub-module 112. The opto-electric conversion sub-module 111 may include circuit components such as a photodiode and an avalanche photodiode, and is used to convert an optical signal into an electrical signal. The electro-optic conversion sub-module 112 may include circuit components such as a laser and a light-emitting diode, and is used to convert an electrical signal into an optical signal. During the process of opto-electric conversion or electro-optic conversion, there may be a mapping relationship between the characteristics of the electrical signal and the characteristics of the optical signal, that is, an electrical signal with a first characteristic can be converted into an optical signal with a second characteristic, or an optical signal with a second characteristic can be converted into an electrical signal with a first characteristic. Among them, the first characteristic of the electrical signal may include but is not limited to the voltage, current, frequency, phase, amplitude, etc. of the electrical signal. The second characteristic of the optical signal may include but is not limited to the intensity, wavelength, phase, polarization, power, etc. of the optical signal. For example, the amplitude of the electrical signal can be mapped to the intensity of the optical signal, and electrical signals with different amplitudes can be converted into optical signals with different intensities. For another example, the frequency of the electrical signal can be mapped to the wavelength of the optical signal. Electrical signals with different frequencies can be converted into optical signals with different wavelengths.

[0063] Specifically, the characteristics of the electrical signal such as voltage, current, frequency, phase, amplitude, etc., and the characteristics of the optical signal such as intensity, wavelength, phase, polarization, power, etc., can be used to encode digital information. Based on the digital information carried by the optical signal or the electrical signal, the mutual conversion between the electrical signal and the optical signal can be performed. For example, assume that a low level is used to represent the digital information 0, a high level is used to represent the digital information 1, and an optical signal with a first brightness is used to represent the digital information 0, and an optical signal with a second brightness is used to represent the digital information 1. Then, based on the digital information carried by the electrical signal, an optical signal can be obtained through conversion, or based on the digital information carried by the optical signal, an electrical signal can be obtained through conversion.

[0064] Referring to Figure 2 , a schematic diagram of the module of the photon arithmetic unit 12 provided in some embodiments of the present application. The photon arithmetic unit 12 may be integrated in an optical integrated circuit and is used for optical computing. As Figure 2As shown, the photon arithmetic unit 12 may include at least one optical path 121 for transmitting optical signals. The optical path 121 may include optical elements such as lenses, prisms, gratings, waveguides, modulators, detectors, etc. After configuring the optical path 121, at least some characteristics of the optical signal can be changed when the optical signal passes through the optical path 121. Among them, configuring the optical path 121 means configuring the number, position, working characteristics, etc. of the optical elements in the optical path 121. For example, changing the voltage or current magnitude of the modulator, increasing or decreasing the number of gratings, or changing the position of the waveguide. Different configurations of the optical path 121 may result in different characteristics of the optical signal changed by the optical path 121. For example, when the voltage magnitude of the modulator in the optical path 121 is set to 0.5 volts, after inputting the optical signal with the first brightness into the optical path 121 and through the transmission of the optical path 121, the photon arithmetic unit 12 can output the optical signal with the second brightness; when the voltage magnitude of the modulator in the optical path 121 is set to 0.45 volts, after inputting the optical signal with the first brightness into the optical path 121 and through the transmission of the optical path 121, the photon arithmetic unit 12 can output the optical signal with the third brightness.

[0065] Based on the working principle of the photon arithmetic unit 12, at least some operation terms in the mathematical operation can be converted into the first optical signal, and the optical path 121 can be configured based on the operation logic. In this way, after inputting the first optical signal into the optical path 121 of the arithmetic unit and through the transmission of the optical path 121, the second optical signal output by the photon arithmetic unit 12 can represent the operation result. For example, when performing the operation of 5 + 4, the addend 5 can be converted into the first optical signal, and then based on the addend 4 and the operation method (i.e., addition), the optical path 121 is configured so that after inputting the first optical signal corresponding to the addend 5 into the optical path 121 and through the transmission of the optical path 121, the brightness of the first optical signal can be enhanced to obtain the second optical signal with the brightness corresponding to the number 9. In this way, the second optical signal output by the photon arithmetic unit 12 can be used as the operation result.

[0066] In the case where the photon arithmetic unit 12 includes multiple optical paths 121, different configurations can be performed on different optical paths 121. In this way, different characteristic changes can be made to the optical signals transmitted in different optical paths 121. For example, after the first optical signal A1 with the first brightness passes through the optical path P1, the photon arithmetic unit 12 can output the second optical signal A2 with the second brightness from the optical path P1; after the first optical signal B1 with the first amplitude passes through the optical path P2, the photon arithmetic unit 12 can output the second optical signal B2 with the second amplitude from the optical path P2, that is, the optical path P1 can change the brightness of the optical signal, and the optical path P2 can change the amplitude of the optical signal. In this way, different operations can be performed in parallel based on different optical paths 121. Of course, it can be understood that the same configuration can also be performed on multiple optical paths 121. In this way, the same operation can be performed in parallel based on multiple optical paths 121.

[0067] Based on the above description, with reference to Figure 3 , it is a schematic flowchart of a model training method provided for some embodiments of this application. Figure 3 In , the model training method may include the following steps:

[0068] Step S301, receive a model training task transmitted using an electrical signal, and extract a matrix multiplication operation from the model training task, where the matrix multiplication operation includes a first matrix and a second matrix.

[0069] Specifically, for a scenario where multiple model training devices execute the model training task in parallel, the multiple model training devices can be connected to a host computer. In the host computer, the model training task can be divided into multiple subtasks, and the host computer can send the multiple subtasks to different model training devices in the form of electrical signals. Each model training device can receive and execute the subtasks sent by the host computer. For example, in the host computer, the model training task can be divided into subtasks B1, B2, and B3 according to the neural network layers, where subtask B1 represents the training task of neural network layers L1 to L3, subtask B2 represents the training task of neural network layers L4 to L5, and subtask B3 represents the training task of neural network layers L6 to L7. In the form of electrical signals, the host computer can send subtask B1 to model training device M1, send subtask B2 to model training device M2, and send subtask B3 to model training device M3. After each model training device receives the model training task, the central processing unit or graphics processing unit in the model training device can extract the matrix multiplication operation from the model training task. It can be understood that since the model training tasks executed by different model training devices are different, the matrix multiplication operations extracted by each model training device may be different.

[0070] For a scenario where all model training tasks are executed by the same model training device. The central processing unit or graphics processing unit in the model training device can extract all matrix multiplication operations involved in the model training process.

[0071] In the matrix multiplication operation, there are at least a first matrix and a second matrix. The first matrix can be regarded as the matrix to be multiplied, and the second matrix can be regarded as the multiplying matrix. The multiplication of the first matrix and the second matrix can be regarded as performing a matrix transformation on the first matrix based on the second matrix to obtain a result matrix. The second matrix defines the linear transformation from the first matrix to the result matrix.

[0072] It should be noted that since the interaction between the first matrix and the second matrix is mutual, in some other embodiments, the multiplication of the first matrix and the second matrix can also be regarded as performing a matrix transformation on the second matrix based on the first matrix to obtain a result matrix. The first matrix defines the linear transformation from the second matrix to the result matrix.

[0073] Step S302: Set the transformation parameters of the photonic arithmetic unit according to the electrical signal corresponding to the second matrix. The transformation parameters represent the matrix transformation to be performed on the first matrix in the matrix multiplication operation.

[0074] In this embodiment, the central processing unit or the graphics processing unit in the model training device can set the transformation parameters of the photonic arithmetic unit based on the extracted second matrix. The process of setting the transformation parameters can be understood as configuring the optical path in the photonic arithmetic unit. Different optical paths in the photonic arithmetic unit can be used to transform different elements in the first matrix. According to the specific values of the elements in the second matrix, each optical path can have its corresponding configuration.

[0075] Step S303: Convert the electrical signal corresponding to the first matrix into a first optical signal, and input the first optical signal into the photonic arithmetic unit to obtain the operation result of the matrix multiplication operation.

[0076] Specifically, the central processing unit or the graphics processing unit in the model training device can input the electrical signal corresponding to the first matrix into the optoelectronic conversion module. The optoelectronic conversion module can convert the electrical signal corresponding to the first matrix into a first optical signal and input the first optical signal into the photonic arithmetic unit with the optical path already configured. The photonic arithmetic unit can convert the first optical signal according to the optical path configuration and output the converted second optical signal. The second optical signal can represent the transformation result of the first matrix.

[0077] Compared with the electronic arithmetic unit that relies on logic gate circuits for bit-by-bit processing when performing matrix multiplication operations, which has the problem of high latency, the photonic arithmetic unit can utilize the parallel propagation characteristics of light to complete matrix multiplication operations at the speed of light. The latency is only determined by the optical path length (nanosecond level) and there is no need to process data bit by bit. Therefore, the operation efficiency can be greatly improved, and thus the problem of low model training efficiency in the related art is solved. In addition, compared with the power consumption of the electronic arithmetic unit that surges with frequency and integration, since the optical signal hardly generates heat during transmission, using the photonic arithmetic unit for matrix multiplication operations can greatly reduce the power consumption.

[0078] Step S304: Perform model training based on the operation result of the matrix multiplication operation.

[0079] Specifically, the central processing unit or the graphics processing unit in the model training device can modify the model parameters or continue with subsequent operations based on the operation result of the matrix multiplication operation.

[0080] In summary, in the technical solutions of some embodiments of the present application, when receiving a model training task transmitted by an electrical signal, the matrix multiplication operation can be extracted from the model training task, and the transformation parameters of the photonic arithmetic unit can be set according to the electrical signal corresponding to the second matrix in the matrix multiplication operation. In this way, after converting the electrical signal corresponding to the first matrix in the matrix multiplication operation into a first optical signal and inputting the first optical signal into the photonic arithmetic unit, the matrix multiplication operation can be completed. Compared with the electronic arithmetic unit that relies on logic gate circuits for bit-by-bit processing when performing matrix multiplication operations and has a high latency problem, the photonic arithmetic unit can complete matrix multiplication operations at the speed of light by utilizing the parallel propagation characteristics of light. The latency is only determined by the optical path length (nanosecond level) and there is no need to process data bit by bit. Therefore, the operation efficiency can be greatly improved, and thus the problem of low model training efficiency in the related art is solved. In addition, compared with the power consumption of the electronic arithmetic unit that surges with frequency and integration, since the optical signal hardly generates heat during transmission, using the photonic arithmetic unit for matrix multiplication operations can greatly reduce the power consumption.

[0081] The following further describes the solution of the present application.

[0082] In some embodiments, considering that electronic computing has advantages such as mature technology, low cost, high integration, and good flexibility, and using an electronic arithmetic unit to perform non-matrix multiplication operations has a relatively small impact on the operation efficiency and model training efficiency. Therefore, the operations in the model training task can be performed based on a combination of optical computing and electronic computing. Specifically, in these embodiments, after receiving a model training task transmitted by an electrical signal, the model training task can be performed according to the following steps:

[0083] 1) Extract the matrix multiplication operation and non-matrix multiplication operation from the model training task. Among them, the non-matrix multiplication operation may include one or more non-matrix terms and the target operation type for the non-matrix terms. The non-matrix terms refer to constants, variables, etc. represented in a non-matrix form. The target operation type may include, but is not limited to, addition operation, subtraction operation, division operation, multiplication operation, statistical operation, conditional operation, etc.

[0084] 2) Based on the above steps S301 to S304, use the photonic arithmetic unit to perform the matrix multiplication operation to obtain the matrix multiplication operation result represented by the optical signal.

[0085] 3) Input the electrical signals corresponding to the non-matrix terms in the non-matrix multiplication operation into an electronic arithmetic unit that supports the target operation type to obtain the operation result of the non-matrix multiplication operation. Specifically, in a central processing unit and a graphics processing unit, electronic arithmetic units that support different operation types can be integrated, such as an electronic arithmetic unit that supports addition operations, an electronic arithmetic unit that supports subtraction operations, etc. Based on the target operation type in the non-matrix multiplication operation, after inputting the electrical signals corresponding to the non-matrix terms into the electronic arithmetic unit that supports the target operation type, the electronic arithmetic unit can output the non-matrix multiplication operation result represented by the electrical signals.

[0086] 4) Perform model training based on the matrix multiplication operation result and the non-matrix multiplication operation result.

[0087] In the above embodiments, using a photonic arithmetic unit to perform matrix multiplication operations can improve the operation efficiency and model training efficiency. Using an electronic arithmetic unit to perform non-matrix multiplication operations can improve the operation stability and reduce the operation cost. By combining optical computing and electronic computing, the advantages of optical computing and electronic computing can be fully utilized, so that while improving the model training efficiency, the cost of model training can be reduced, and the stability and flexibility of model training can be improved.

[0088] Further, in some embodiments, the non-matrix multiplication operation further includes matrix terms that perform non-multiplication operations, such as matrix terms that perform addition operations or subtraction operations. For the operations of these matrix terms, either optical computing or electronic computing can be used, and the present application does not limit this.

[0089] In some embodiments, there may be a dependency relationship between the matrix multiplication operation and the non-matrix multiplication operation. The so-called dependency relationship means that the execution of the non-matrix multiplication operation depends on the operation result of the matrix multiplication operation, or the execution of the matrix multiplication operation depends on the operation result of the non-matrix multiplication operation. For example, in the above expressions (5) and (6), it is necessary to perform the non-matrix multiplication operation in expression (5) based on the matrix multiplication operation result of expression (6). .

[0090] In view of this, when extracting the matrix multiplication operation and the non-matrix multiplication operation from the model training task, the dependency relationship between the matrix multiplication operation and the non-matrix multiplication operation can be analyzed, and based on the dependency relationship, the execution order of the matrix multiplication operation and the non-matrix multiplication operation can be determined. Specifically, if the non-matrix multiplication operation depends on the operation result of the matrix multiplication operation, the non-matrix multiplication operation is performed after the matrix multiplication operation. If the matrix multiplication operation depends on the operation result of the non-matrix multiplication operation, the matrix multiplication operation is performed after the non-matrix multiplication operation. In this way, it can be ensured that each operation in the model training task can be executed in an orderly manner, preventing the problem that the operation task fails to execute due to not obtaining the operation result on which the operation task depends.

[0091] In some embodiments, when a non-matrix multiplication operation depends on the operation result of a matrix multiplication operation, it is required that the operation result of the matrix multiplication operation be represented by an electrical signal. However, in practice, the operation result of the matrix multiplication operation output by the photonic arithmetic unit is represented by an optical signal, which cannot meet the actual requirements. In view of this, after the first optical signal is input into the photonic arithmetic unit, the second optical signal output by the photonic arithmetic unit can also be converted into an electrical signal, and the converted electrical signal is used as the operation result of the matrix multiplication operation. In this way, the electrical signal representing the operation result of the matrix multiplication operation can be directly used as an input item of the non-matrix multiplication operation and participate in the non-matrix multiplication operation.

[0092] Of course, it can be understood that if there is a dependency relationship between matrix multiplication operations, it is not necessary to convert the second optical signal output by the photonic arithmetic unit into an electrical signal. For example, assuming that matrix multiplication operation a depends on the operation result of matrix multiplication operation b, after the photonic arithmetic unit outputs the operation result of matrix multiplication operation b in the form of a second optical signal, the second optical signal can be directly used as the input of matrix multiplication operation a. In this case, it is not necessary to convert the second optical signal output by the photonic arithmetic unit into an electrical signal. In this way, the signal conversion time can be reduced and the model training efficiency can be improved.

[0093] In some embodiments, each row of the first matrix can be used as a row vector respectively. The first matrix can include at least one row vector, and the first optical signal can include at least one set of sub-optical signals, and the at least one set of sub-optical signals corresponds to the row vectors of the first matrix one by one. The above-mentioned input of the first optical signal into the photonic arithmetic unit to obtain the operation result of the matrix multiplication operation includes:

[0094] Sequentially input the sub-optical signals corresponding to each row vector into the photonic arithmetic unit to obtain the operation results of each row vector;

[0095] Summarize the operation results of each row vector to obtain the operation result of the matrix multiplication operation.

[0096] For example, assuming that the first matrix includes three row vectors a1, a2, and a3, the sub-optical signal corresponding to the row vector a1 can be input into the photonic arithmetic unit to obtain the operation result q1 corresponding to the row vector a1, the sub-optical signal corresponding to the row vector a2 can be input into the photonic arithmetic unit to obtain the operation result q2 corresponding to the row vector a2, and the sub-optical signal corresponding to the row vector a3 can be input into the photonic arithmetic unit to obtain the operation result q3 corresponding to the row vector a3. Finally, summarize the operation results q1, q2, and q3 to obtain the operation result of the matrix multiplication operation.

[0097] In the above embodiments, determining the operation result of the matrix multiplication operation based on the row vector can reduce the design difficulty of the photonic arithmetic unit.

[0098] In some embodiments, a photonic arithmetic unit may be designed based on a Mach–Zehnder interferometer (MZI). A Mach–Zehnder interferometer is an optical silicon-based device based on the principle of light interference, and its core function is to be able to precisely control the amplitude and phase of optical signals, thereby completing complex mathematical operations. Referring to Figure 4 , a schematic diagram of a module of a Mach–Zehnder interferometer provided in some embodiments of this application. Figure 4 In, the Mach–Zehnder interferometer includes a first phase-shifting arm 41, a second phase-shifting arm 42, an input coupler 43, and an output coupler 44. Among them, the phase-shifting arm is the optical path. The first phase-shifting arm 41 includes one or more phase modulators 45. The second phase-shifting arm 42 can be used as a reference arm and may not include a phase modulator 45. The input coupler 43 can be used as a beam splitter for optical signals, and the output coupler 44 can be used as a beam combiner for optical signals. Both the input coupler 43 and the output coupler 44 can be referred to as 2×2 couplers. Based on the above structure, the Mach–Zehnder interferometer allows adjustment of the optical path difference to achieve interference of different phases. Specifically, when the input optical signal passes through the Mach–Zehnder interferometer, it will be split into two beams by the input coupler 43, and the two optical signals are respectively transmitted along the first phase-shifting arm 41 and the second phase-shifting arm 42. During the transmission process, the phase modulator 45 in the first phase-shifting arm 41 will change the phase of the optical signal based on the configuration. After passing through the output coupler 44, the two beams of light will recombine spatially to form a new optical signal, which can be considered as a linear combination of the input optical signal. Based on the above principle, the Mach–Zehnder interferometer can perform matrix operations. Specifically, the 2nd-order equivalent matrix of a Mach–Zehnder interferometer unit with two-port input and two-port output can be as shown in expression (7):

[0099] (7)

[0100] Wherein, , represent the input optical signal, and can represent the phase changes of the input optical signal , . , represent the output optical signal.

[0101] Figure 4 The Mach–Zehnder interferometer in can be used as a basic Mach–Zehnder interferometer unit. By topologically cascading Mach–Zehnder interferometer units, unitary matrices of any scale can be realized. For example, assuming that each rectangle in Figure 5 and Figure 6 is respectively used as a Mach–Zehnder interferometer unit, according toFigure 5 The triangular array arrangement shown or Figure 6 The rectangular array arrangement shown, topologically cascading the Mach-Zehnder interferometer units, can realize unitary matrices of any scale.

[0102] Based on the above principle, in combination with referring to Figure 7 , a schematic diagram of the internal module of the photonic arithmetic unit provided by some embodiments of the present application. Figure 7 In, the photonic arithmetic unit includes a first network, a second network, and an attenuator array. Among them, the first network represents the left unitary matrix obtained after performing singular value decomposition on the matrix, the second network represents the complex conjugate transpose of the right unitary matrix obtained after performing singular value decomposition on the matrix, and the attenuator array represents the rectangular diagonal matrix obtained after performing singular value decomposition on the matrix. Since any-scale unitary matrices can be realized by topologically cascading Mach-Zehnder interferometer units, the above first network, second network, and attenuator array can be realized by topologically cascading Mach-Zehnder interferometer units.

[0103] Based on the first network, second network, and attenuator array inside the photonic arithmetic unit, setting the transformation parameters of the photonic arithmetic unit according to the electrical signal corresponding to the second matrix may include:

[0104] Performing singular value decomposition on the second matrix to obtain the left unitary matrix, the rectangular diagonal matrix, and the complex conjugate transpose of the right unitary matrix;

[0105] Setting the transformation parameters of the photonic arithmetic unit according to the electrical signal corresponding to the left unitary matrix, the electrical signal corresponding to the rectangular diagonal matrix, and the electrical signal corresponding to the complex conjugate transpose of the right unitary matrix.

[0106] In this way, it is equivalent to realizing the second matrix inside the photonic arithmetic unit.

[0107] In the above embodiments, the second matrix is realized inside the photonic arithmetic unit by topologically cascading Mach-Zehnder interferometer units, enhancing the feasibility of optical computing.

[0108] In some embodiments, the first network and the second network each include multiple optical paths for transmitting the first optical signal, and the transformation parameters of the photonic arithmetic unit include the attenuation coefficients of the attenuator array and the phase differences of different optical paths in the first network and the second network;

[0109] The above setting of the transformation parameters of the photonic arithmetic unit according to the electrical signal corresponding to the left unitary matrix, the electrical signal corresponding to the rectangular diagonal matrix, and the electrical signal corresponding to the complex conjugate transpose of the right unitary matrix includes:

[0110] Setting the phase differences of different optical paths in the first network according to the electrical signal corresponding to the left unitary matrix;

[0111] Set the phase differences of different optical paths in the second network according to the electrical signals corresponding to right unitary matrices;

[0112] Set the attenuation coefficients of the attenuator array according to the electrical signals corresponding to rectangular diagonal matrices.

[0113] Thus, the related description of the model training method of this application is completed.

[0114] Corresponding to the model training method, this application also provides a model training device. Referring to Figure 8 together, it is a schematic diagram of the modules of the model training device 800 provided in some embodiments of this application. Figure 8 In , the model training device 800 includes a central processing unit 81, an optoelectronic conversion module 82, a graphics processing unit 83, and an optical integrated circuit 84. The graphics processing unit 83 includes an electronic arithmetic unit 831, and the optical integrated circuit 84 includes a photonic arithmetic unit 841.

[0115] One of the central processing unit 81 and the graphics processing unit 83 is used to receive a model training task transmitted using electrical signals, extract the matrix multiplication operation including a first matrix and a second matrix from the model training task, input the electrical signals corresponding to the first matrix into the optoelectronic conversion module 82, and set the transformation parameters of the photonic arithmetic unit 841 according to the electrical signals corresponding to the second matrix. The transformation parameters represent the matrix transformation to be performed on the first matrix in the matrix multiplication operation;

[0116] The optoelectronic conversion module 82 is used to convert the electrical signals corresponding to the first matrix into first optical signals and input the first optical signals into the photonic arithmetic unit 841;

[0117] The photonic arithmetic unit 841 is used to transform the first optical signals according to the transformation parameters to obtain the operation result of the matrix multiplication operation;

[0118] One of the central processing unit 81 and the graphics processing unit 83 performs model training according to the operation result of the matrix multiplication operation.

[0119] In some embodiments, one of the central processing unit 81 and the graphics processing unit 83 is further used to perform the following operations:

[0120] Extract non-matrix multiplication operations from the model training task. The non-matrix multiplication operations include one or more non-matrix terms and the target operation type for the non-matrix terms;

[0121] Input the electrical signals corresponding to the non-matrix terms into the electronic arithmetic unit 831 that supports the target operation type to obtain the operation result of the non-matrix multiplication operation;

[0122] Perform model training according to the operation result of the non-matrix multiplication operation.

[0123] In some embodiments, the first matrix includes at least one row vector, the first optical signal includes at least one set of sub-optical signals, and at least one set of sub-optical signals corresponds one-to-one with the row vectors of the first matrix;

[0124] The optoelectronic conversion module 82 is specifically configured to sequentially input the sub-optical signals corresponding to each row vector into the photonic arithmetic unit 841;

[0125] The photonic arithmetic unit 841 is specifically configured to perform transformations on the sub-optical signals corresponding to each row vector respectively according to the transformation parameters to obtain the operation results of each row vector;

[0126] One of the central processing unit 81 and the graphics processing unit 83 is specifically configured to summarize the operation results of each row vector to obtain the operation result of the matrix multiplication operation.

[0127] In some embodiments, one of the central processing unit 81 and the graphics processing unit 83 is further configured to perform the following operations:

[0128] Perform singular value decomposition on the second matrix to obtain the left unitary matrix, the rectangular diagonal matrix, and the complex conjugate transpose of the right unitary matrix;

[0129] Set the transformation parameters of the photonic arithmetic unit 841 according to the electrical signals corresponding to the left unitary matrix, the electrical signals corresponding to the rectangular diagonal matrix, and the electrical signals corresponding to the complex conjugate transpose of the right unitary matrix.

[0130] In some embodiments, the photonic arithmetic unit 841 includes a first network, a second network, and an attenuator array. The first network and the second network respectively include multiple optical paths for transmitting the first optical signal. The transformation parameters of the photonic arithmetic unit 841 include the attenuation coefficients of the attenuator array and the phase differences of different optical paths in the first network and the second network;

[0131] One of the central processing unit 81 and the graphics processing unit 83 is further configured to perform the following operations:

[0132] Set the phase differences of different optical paths in the first network according to the electrical signals corresponding to the left unitary matrix;

[0133] Set the phase differences of different optical paths in the second network according to the electrical signals corresponding to the right unitary matrix;

[0134] Set the attenuation coefficients of the attenuator array according to the electrical signals corresponding to the rectangular diagonal matrix.

[0135] Regarding the working principle and beneficial effects of the model training device, reference may be made to the relevant description of the model training method, which will not be elaborated here.

[0136] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0137] With reference to Figure 9 , an embodiment of the present application further provides an electronic device, including a memory 10 and a processor 20. A computer program is stored in the memory 10, and the processor 20 is configured to run the computer program to execute the steps in any of the above embodiments of the state data saving method.

[0138] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps in any of the above embodiments of the state data saving method when running.

[0139] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: USB flash drive, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disk, magnetic disk or optical disk, etc., various media that can store computer programs.

[0140] An embodiment of the present application further provides a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above embodiments of the state data saving method.

[0141] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above embodiments of the state data saving method.

[0142] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0143] The above has introduced in detail a model training method, apparatus, device, and storage medium provided by the present application. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A model training method, characterized in that, The method includes: Receiving a model training task transmitted using an electrical signal, and extracting a matrix multiplication operation and a non-matrix multiplication operation from the model training task, where the matrix multiplication operation includes a first matrix and a second matrix, and the non-matrix multiplication operation includes one or more non-matrix terms and a target operation type for the non-matrix terms; Inputting the electrical signal corresponding to the non-matrix term into an electronic arithmetic unit that supports the target operation type to obtain the operation result of the non-matrix multiplication operation; Setting transformation parameters of a photonic arithmetic unit according to the electrical signal corresponding to the second matrix, where the transformation parameters represent a matrix transformation to be performed on the first matrix in the matrix multiplication operation; Converting the electrical signal corresponding to the first matrix into a first optical signal, and inputting the first optical signal into the photonic arithmetic unit to obtain the operation result of the matrix multiplication operation; Performing model training according to the operation results of the matrix multiplication operation and the non-matrix multiplication operation.

2. The method according to claim 1, wherein When extracting the matrix multiplication operation and the non-matrix multiplication operation from the model training task, the method further includes: Analyzing the dependency relationship between the matrix multiplication operation and the non-matrix multiplication operation; Based on the dependency relationship, determining the execution order of the matrix multiplication operation and the non-matrix multiplication operation, where if the non-matrix multiplication operation depends on the operation result of the matrix multiplication operation, the non-matrix multiplication operation is executed after the matrix multiplication operation, and if the matrix multiplication operation depends on the operation result of the non-matrix multiplication operation, the matrix multiplication operation is executed after the non-matrix multiplication operation.

3. The method according to claim 1, wherein The first matrix includes at least one row vector, the first optical signal includes at least one set of sub-optical signals, and the at least one set of sub-optical signals corresponds one-to-one to the row vectors of the first matrix; The step of inputting the first optical signal into the photonic arithmetic unit to obtain the operation result of the matrix multiplication operation includes: Sequentially inputting the sub-optical signals corresponding to each row vector into the photonic arithmetic unit to obtain the operation results of each row vector; Summarizing the operation results of each row vector to obtain the operation result of the matrix multiplication operation.

4. The method according to claim 1, wherein The step of setting the transformation parameters of the photonic arithmetic unit according to the electrical signal corresponding to the second matrix includes: Performing singular value decomposition on the second matrix to obtain a left unitary matrix, a rectangular diagonal matrix, and the complex conjugate transpose of a right unitary matrix; Setting the transformation parameters of the photonic arithmetic unit according to the electrical signal corresponding to the left unitary matrix, the electrical signal corresponding to the rectangular diagonal matrix, and the electrical signal corresponding to the complex conjugate transpose of the right unitary matrix.

5. The method according to claim 4, wherein The photonic arithmetic unit includes a first network, a second network, and an attenuator array. The first network and the second network respectively include multiple optical paths for transmitting the first optical signal. The transformation parameters of the photonic arithmetic unit include the attenuation coefficients of the attenuator array and the phase differences of different optical paths in the first network and the second network; The step of setting the transformation parameters of the photonic arithmetic unit according to the electrical signal corresponding to the left unitary matrix, the electrical signal corresponding to the rectangular diagonal matrix, and the electrical signal corresponding to the complex conjugate transpose of the right unitary matrix includes: Set the phase differences of different optical paths in the first network according to the electrical signal corresponding to the left unitary matrix; Set the phase differences of different optical paths in the second network according to the electrical signal corresponding to the right unitary matrix; Set the attenuation coefficients of the attenuator array according to the electrical signal corresponding to the rectangular diagonal matrix.

6. The method according to claim 1, wherein The photon arithmetic unit is configured to convert the first optical signal according to the transformation parameters and output a second optical signal obtained by the conversion; Inputting the first optical signal into the photon arithmetic unit to obtain the operation result of the matrix multiplication operation includes: Convert the second optical signal output by the photon arithmetic unit into an electrical signal, and use the converted electrical signal as the operation result of the matrix multiplication operation.

7. A model training device, characterized in that, The model training device includes a central processing unit, an optoelectronic conversion module, a graphics processing unit, and an optical integrated circuit. The graphics processing unit includes an electronic arithmetic unit, and the optical integrated circuit includes a photon arithmetic unit, where: One of the central processing unit and the graphics processing unit is configured to receive a model training task transmitted using an electrical signal, extract a non-matrix multiplication operation including a non-matrix term and a target operation type from the model training task, and input the electrical signal corresponding to the non-matrix term into the electronic arithmetic unit supporting the target operation type to obtain the operation result of the non-matrix multiplication operation, and extract a matrix multiplication operation including a first matrix and a second matrix from the model training task, and input the electrical signal corresponding to the first matrix into the optoelectronic conversion module, and set the transformation parameters of the photon arithmetic unit according to the electrical signal corresponding to the second matrix, where the transformation parameters represent the matrix transformation to be performed on the first matrix in the matrix multiplication operation; The optoelectronic conversion module is configured to convert the electrical signal corresponding to the first matrix into a first optical signal and input the first optical signal into the photon arithmetic unit; The photon arithmetic unit is configured to transform the first optical signal according to the transformation parameters to obtain the operation result of the matrix multiplication operation; One of the central processing unit and the graphics processing unit performs model training according to the operation results of the matrix multiplication operation and the non-matrix multiplication operation.

8. The device according to claim 7, characterized in that, The first matrix includes at least one row vector, the first optical signal includes at least one group of sub-optical signals, and the at least one group of sub-optical signals corresponds one-to-one with the row vectors of the first matrix; The optoelectronic conversion module is specifically configured to sequentially input the sub-optical signals corresponding to each row vector into the photon arithmetic unit; The photon arithmetic unit is specifically configured to perform transformations on the sub-optical signals corresponding to each row vector respectively according to the transformation parameters to obtain the operation results of each row vector; One of the central processing unit and the graphics processing unit is specifically configured to aggregate the operation results of each row vector to obtain the operation result of the matrix multiplication operation.

9. The device according to claim 7, characterized in that, One of the central processing unit and the graphics processing unit is further configured to perform the following operations: Perform singular value decomposition on the second matrix to obtain the complex conjugate transpose of the left unitary matrix, the rectangular diagonal matrix, and the right unitary matrix; Set the transformation parameters of the photon arithmetic unit according to the electrical signal corresponding to the left unitary matrix, the electrical signal corresponding to the rectangular diagonal matrix, and the electrical signal corresponding to the complex conjugate transpose of the right unitary matrix.

10. The device according to claim 9, characterized in that The photon arithmetic unit includes a first network, a second network, and an attenuator array. The first network and the second network respectively include multiple optical paths for transmitting the first optical signal. The transformation parameters of the photon arithmetic unit include the attenuation coefficients of the attenuator array and the phase differences of different optical paths in the first network and the second network. One of the central processing unit and the graphics processing unit is further configured to perform the following operations: Set the phase differences of different optical paths in the first network according to the electrical signal corresponding to the left unitary matrix. Set the phase differences of different optical paths in the second network according to the electrical signal corresponding to the right unitary matrix. Set the attenuation coefficients of the attenuator array according to the electrical signal corresponding to the rectangular diagonal matrix.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

12. An electronic device, characterized in that, The electronic device includes a processor and a memory. The memory is used to store a computer program, which, when executed by the processor, implements the method according to any one of claims 1 to 6.

13. A computer program product, comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images