Model training method and device, storage medium and program product

Abstract

The invention discloses a model training method and device, a storage medium and a program product, and relates to the technical field of artificial intelligence computation.The method comprises the steps that when a model training task transmitted through an electric signal is received, matrix multiplication can be extracted from the model training task, and according to the electric signal corresponding to a second matrix in the matrix multiplication, the second matrix is obtained; and setting transformation parameters of the photon arithmetic unit. Therefore, after the electric signal corresponding to the first matrix in the matrix multiplication operation is converted into the first optical signal, the first optical signal is input into the photon arithmetic unit, and the matrix multiplication operation can be completed. The photon arithmetic unit can complete matrix multiplication operation under the light velocity by utilizing the parallel propagation characteristic of light, delay is only determined by the optical path length (nanosecond level), and data bits do not need to be processed one by one, so that the operation efficiency can be greatly improved, and the problem of low model training efficiency in related technologies is solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence computing technology, and particularly to a model training method, device, storage medium, and program product. Background Art

[0002] Model training is a key link in the learning of artificial intelligence algorithms. During some model training processes, a large amount of pre-annotated training data needs to be input into the model. Based on the training data, the model can output prediction results. Based on the difference between the true annotation and the prediction result, the model parameters can be adjusted, so as to improve the prediction accuracy of the model. Currently, model training involves a large amount of operations, and these operations are time-consuming, reducing the model training efficiency. Summary of the Invention

[0003] This application provides a model training method, a model training device, an electronic device, a computer-readable storage medium, and a computer program product to at least solve the problem of low model training efficiency in related technologies.

[0004] This application provides a model training method, the method includes: Receiving a model training task transmitted using an electrical signal, and extracting a matrix multiplication operation from the model training task, the matrix multiplication operation including a first matrix and a second matrix; Setting transformation parameters of a photonic arithmetic unit according to the electrical signal corresponding to the second matrix, the transformation parameters characterizing a matrix transformation to be performed on the first matrix in the matrix multiplication operation; Converting the electrical signal corresponding to the first matrix into a first optical signal, and inputting the first optical signal into the photonic arithmetic unit to obtain an operation result of the matrix multiplication operation; Performing model training according to the operation result of the matrix multiplication operation.

[0005] This application also provides a model training device, the model training device includes a central processing unit, an optoelectronic conversion module, a graphics processing unit, and an optical integrated circuit, the graphics processing unit includes an electronic arithmetic unit, and the optical integrated circuit includes a photonic arithmetic unit, wherein: One of the central processing unit and the graphics processing unit is used to receive a model training task transmitted using an electrical signal, extract a matrix multiplication operation including a first matrix and a second matrix from the model training task, input the electrical signal corresponding to the first matrix into the optoelectronic conversion module, and set transformation parameters of the photonic arithmetic unit according to the electrical signal corresponding to the second matrix, the transformation parameters characterizing a matrix transformation to be performed on the first matrix in the matrix multiplication operation; The photoelectric conversion module is used to convert the electrical signal corresponding to the first matrix into a first optical signal and input the first optical signal into the photonic arithmetic unit; The photonic arithmetic unit is used to transform the first optical signal according to the transformation parameter to obtain the operation result of the matrix multiplication operation; One of the central processing unit and the graphics processing unit performs model training according to the operation result of the matrix multiplication operation.

[0006] This application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of the above model training method when executing the computer program.

[0007] This application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above model training method are implemented.

[0008] This application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the above model training method are implemented.

[0009] In the technical solutions of some embodiments of this application, when receiving a model training task transmitted by an electrical signal, the matrix multiplication operation can be extracted from the model training task, and the transformation parameter of the photonic arithmetic unit can be set according to the electrical signal corresponding to the second matrix in the matrix multiplication operation. In this way, after converting the electrical signal corresponding to the first matrix in the matrix multiplication operation into a first optical signal and inputting the first optical signal into the photonic arithmetic unit, the matrix multiplication operation can be completed. Compared with the electronic arithmetic unit that depends on the logic gate circuit to process bit by bit when performing matrix multiplication operations, there is a problem of high latency. The photonic arithmetic unit can utilize the parallel propagation characteristics of light to complete the matrix multiplication operation at the speed of light. The latency is only determined by the optical path length (nanosecond level) and there is no need to process data bit by bit. Therefore, the operation efficiency can be greatly improved, and the problem of low model training efficiency in the related art is solved. Description of the Drawings

[0010] To more clearly illustrate the embodiments of this application, the drawings required for the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0011] Figure 1 It is a module schematic diagram of the photoelectric conversion module provided by some embodiments of this application; Figure 2 It is a module schematic diagram of the photonic arithmetic unit provided by some embodiments of this application; Figure 3 Flow schematic diagram of the model training method provided for some embodiments of the present application; Figure 4 Module schematic diagram of the Mach-Zehnder interferometer provided for some embodiments of the present application; Figure 5 Topological cascading mode of the Mach-Zehnder interferometer provided for some embodiments of the present application; Figure 6 Topological cascading mode of the Mach-Zehnder interferometer provided for other embodiments of the present application; Figure 7 Internal module schematic diagram of the photonic arithmetic unit provided for some embodiments of the present application; Figure 8 Module schematic diagram of the model training device provided for some embodiments of the present application; Figure 9 Module schematic diagram of the electronic device provided for some embodiments of the present application. Detailed implementation manners

[0012] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0013] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0014] In order to enable those skilled in the art of the present technology to better understand the solution of the present application, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0015] Operations during model training can include matrix multiplication and non-matrix multiplication. Matrix multiplication refers to multiplication involving matrix items, such as multiplication between the model's weight matrix and feature vector. Non-matrix multiplication refers to operations that do not involve matrix items or non-multiplication involving matrix items. For example, addition, subtraction, division, statistical operations, and conditional operations involving matrix items or non-matrix items. Another example is multiplication between non-matrix items. For ease of understanding, the training process of a binary logistic regression model under a supervised learning framework is used as an example to illustrate.

[0016] Assume that the training set D of the binary logistic regression model is Among them, the data points , represents the data point The superscript T indicates transposing the eigenvector, that is, transposing the M-dimensional row vector into a column vector. The value of i is an integer between 1 and N (including 1 and N), and the values ​​of M and N are integers greater than 0. For data points The real annotation represents the data point The actual category to which it belongs. For example, The value of can be 0 or 1. =0, indicating a data point Actually belongs to category 0; =1, indicating a data point Actually belongs to category 1.

[0017] To train a binary logistic regression model, we need to transform the data points The feature vector of is input into the binary logistic regression model, and the binary logistic regression model predicts the data point The category to which it belongs. If the prediction result is different from the data point The same as the true label of the data point, indicating that the prediction of the binary logistic regression model is accurate; if the prediction result is the same as the data point The actual annotations are different, and the model parameters of the binary logistic regression model can be adjusted to make the prediction results consistent with the data points. The actual markings should be as similar as possible.

[0018] Specifically, during the model training process, the data points The eigenvector of After inputting the binary logistic regression model, the binary logistic regression model can output data points The probability of belonging to each category. Among them, the category with the highest probability can be used as the prediction result of the binary logistic regression model. For example, suppose the category probabilities output by the binary logistic regression model are as follows: =0 has a probability of 0.85 The probability of = 1 is 0.15 That is, the data point The probability of belonging to class 0 is 0.85, and the data point The probability of belonging to class 1 is 0.15. Since the probability of class 0 is the largest, = 0 can be used as the prediction result of the binary logistic regression model. At this time, if the true label of the data point is also 0, it means that the prediction result of the binary logistic regression model is accurate; if the true label of the data point is 1, it means that the prediction result of the binary logistic regression model is inaccurate, and the model parameters of the binary logistic regression model can be adjusted to improve the accuracy of the prediction result. Based on the above description, in the binary logistic regression model, the probabilities of the data point belonging to each class can be shown as expressions (1) and (2).

[0019] (1) (2) Among them, represents the probability that the data point belongs to class 1, represents the probability that the data point belongs to class 0, represents the transpose of the feature vector of the data point , represents the model parameter. During the model training process, the model parameter w can be adjusted according to the true label of the data point and the prediction result of the binary logistic regression model.

[0020] In order to obtain the optimal model parameters, an objective function as shown in expression (3) can be constructed.

[0021] (3) Among them, , . The process of model training is to find the optimal model parameter corresponding to the minimum of the objective function .

[0022] Since the objective function has no closed-form solution (that is, when takes the minimum value, the optimal model parameter cannot be explicitly expressed), the gradient descent algorithm can be used to obtain the optimal model parameter . Specifically, assume that the gradient vector of the objective function is The gradient calculation formula of the objective function can be shown as in Expression (4).

[0023] (4) Wherein, , represents the j-th element in the data point , represents the j-th element in the gradient vector .

[0024] Based on the above Expressions (1) to (4), the training process of the binary logistic regression model includes the following steps 1) to 5).

[0025] 1) Set the model training accuracy , the step size factor and the initial model parameters , and set the initial value of the iteration number t = 0.

[0026] 2) According to the above Expression (4), calculate all the gradient information , and based on the gradient information , form the gradient vector . Wherein, represents the j-th element in the gradient vector in the t-th iteration.

[0027] 3) If , then take the model parameters obtained in the t-th iteration as the optimal model parameters , and substitute them into the objective function in Expression (3) to calculate the corresponding value, and then end the process.

[0028] 4) If , then update the model parameters according to the iteration rule , and execute step 5).

[0029] 5) Increment the iteration number t by 1, and repeat the above steps 2) to 4) until the optimal model parameters are obtained.

[0030] In the above training process, the gradient needs to be calculated multiple times, and the gradient calculation can be shown as in Expression (5).

[0031] (5) Wherein, X represents the data matrix composed of the feature vectors of each data point. In the data matrix X, each row represents the feature vector of a data point, represents the transpose of the data matrix X, A vector composed of the differences between the predicted probabilities and the true labels of each data point.

[0032] Also, since , before calculating expression (5), it is necessary to first calculate the inner product of the feature vector of the data point and the model parameters based on expression (6).

[0033] (6) Substitute the calculation result of expression (5) into expression (4), and the gradient vector can be calculated.

[0034] Based on the training process of the above binary classification logistic regression model, it can be seen that during the model training process, a large number of matrix multiplication operations are involved, such as the above expressions (5) and (6), and a large number of non-matrix multiplication operations are also involved, such as in the above expression (5).

[0035] Currently, all operations during the model training process are implemented based on the electronic arithmetic units integrated in the processor. Among them, the processor can include but not limited to Graphics Processing Unit (GPU), Central Processing Unit (CPU), etc. The electronic arithmetic unit can include analog circuits or digital circuits. Analog circuits can perform mathematical operations using continuously varying physical quantities (such as voltage and current). Digital circuits can perform mathematical operations using discrete digital signals (such as binary signals). This operation method based on the electronic arithmetic unit has advantages such as mature technology, low cost, high integration, and good flexibility, but when performing some operations, the operation efficiency is relatively low, which will in turn lead to the problem of low model training efficiency. For example, when the electronic arithmetic unit performs matrix multiplication operations, it depends on the logic gate circuit to process bit by bit, resulting in a high latency problem, which greatly reduces the operation efficiency and in turn reduces the model training efficiency. In addition, the power consumption of the electronic arithmetic unit surges with the frequency and integration, and when using the electronic arithmetic unit to perform matrix multiplication operations, the power consumption is large.

[0036] In view of this, the present application provides a model training method, which can solve the problem of low model training efficiency in some technologies. The model training method can be applied to a model training device. Among them, the model training device can include but not limited to servers, inference boards, etc. Modules such as a central processing unit, a graphics processing unit, a photoelectric conversion module, and a photonic arithmetic unit can be set in the model training device. For the convenience of subsequent scheme elaboration, the photoelectric conversion module and the photonic arithmetic unit will be described first below.

[0037] Refer to Figure 1, which is a schematic diagram of the optoelectronic conversion module 11 provided for some embodiments of the present application. Figure 1 In it, the optoelectronic conversion module 11 includes an optical-electric conversion sub-module 111 and an electric-optical conversion sub-module 112. The optical-electric conversion sub-module 111 may include circuit components such as photodiodes and avalanche photodiodes, and is used to convert optical signals into electrical signals. The electric-optical conversion sub-module 112 may include circuit components such as lasers and light-emitting diodes, and is used to convert electrical signals into optical signals. During the process of optical-electric conversion or electric-optical conversion, there may be a mapping relationship between the characteristics of the electrical signal and the characteristics of the optical signal, that is, an electrical signal with a first characteristic can be converted into an optical signal with a second characteristic, or an optical signal with a second characteristic can be converted into an electrical signal with a first characteristic. Among them, the first characteristic of the electrical signal may include but is not limited to the voltage, current, frequency, phase, amplitude, etc. of the electrical signal. The second characteristic of the optical signal may include but is not limited to the intensity, wavelength, phase, polarization, power, etc. of the optical signal. For example, the amplitude of the electrical signal can be mapped to the intensity of the optical signal, and electrical signals with different amplitudes can be converted into optical signals with different intensities. For another example, the frequency of the electrical signal can be mapped to the wavelength of the optical signal. Electrical signals with different frequencies can be converted into optical signals with different wavelengths.

[0038] Specifically, the characteristics such as the voltage, current, frequency, phase, amplitude, etc. of the electrical signal, and the characteristics such as the intensity, wavelength, phase, polarization, power, etc. of the optical signal can be used to encode digital information. Based on the digital information carried by the optical signal or the electrical signal, the conversion between the electrical signal and the optical signal can be performed. For example, assuming that a low level is used to represent the digital information 0, a high level is used to represent the digital information 1, and an optical signal with a first brightness is used to represent the digital information 0, and an optical signal with a second brightness is used to represent the digital information 1. Then, based on the digital information carried by the electrical signal, an optical signal can be obtained through conversion, or based on the digital information carried by the optical signal, an electrical signal can be obtained through conversion.

[0039] Refer to Figure 2 , which is a schematic diagram of the photon arithmetic unit 12 provided for some embodiments of the present application. The photon arithmetic unit 12 can be integrated in an optical integrated circuit and is used for optical computing. As Figure 2As shown, the photon arithmetic unit 12 may include at least one optical path 121 for transmitting optical signals. The optical path 121 may include optical elements such as lenses, prisms, gratings, waveguides, modulators, detectors, etc. After configuring the optical path 121, at least some characteristics of the optical signal can be changed when the optical signal passes through the optical path 121. Among them, configuring the optical path 121 means configuring the number, position, working characteristics, etc. of the optical elements in the optical path 121. For example, changing the voltage or current magnitude of the modulator, increasing or decreasing the number of gratings, or changing the position of the waveguide. By making different configurations of the optical path 121, the characteristics of the optical signal changed by the optical path 121 can be different. For example, when the voltage magnitude of the modulator in the optical path 121 is set to 0.5 volts, after inputting the optical signal with the first brightness into the optical path 121 and through the transmission of the optical path 121, the photon arithmetic unit 12 can output the optical signal with the second brightness; when the voltage magnitude of the modulator in the optical path 121 is set to 0.45 volts, after inputting the optical signal with the first brightness into the optical path 121 and through the transmission of the optical path 121, the photon arithmetic unit 12 can output the optical signal with the third brightness.

[0040] Based on the working principle of the photon arithmetic unit 12, at least some operation terms in the mathematical operation can be converted into the first optical signal, and the optical path 121 can be configured based on the operation logic. In this way, after inputting the first optical signal into the optical path 121 of the arithmetic unit and through the transmission of the optical path 121, the second optical signal output by the photon arithmetic unit 12 can represent the operation result. For example, when performing the operation of 5 + 4, the addend 5 can be converted into the first optical signal, and then based on the addend 4 and the operation method (i.e., addition), the optical path 121 is configured so that after inputting the first optical signal corresponding to the addend 5 into the optical path 121 and through the transmission of the optical path 121, the brightness of the first optical signal can be enhanced to obtain the second optical signal with the brightness corresponding to the number 9. In this way, the second optical signal output by the photon arithmetic unit 12 can be used as the operation result.

[0041] When the photon arithmetic unit 12 includes multiple optical paths 121, different configurations can be made for different optical paths 121. In this way, different characteristic changes can be made to the optical signals transmitted in different optical paths 121. For example, after the first optical signal A1 with the first brightness passes through the optical path P1, the photon arithmetic unit 12 can output the second optical signal A2 with the second brightness from the optical path P1; after the first optical signal B1 with the first amplitude passes through the optical path P2, the photon arithmetic unit 12 can output the second optical signal B2 with the second amplitude from the optical path P2, that is, the optical path P1 can change the brightness of the optical signal, and the optical path P2 can change the amplitude of the optical signal. In this way, based on different optical paths 121, different operations can be executed in parallel. Of course, it can be understood that the same configuration can also be made for multiple optical paths 121. In this way, the same operation can be executed in parallel based on multiple optical paths 121.

[0042] Based on the above description, with reference to Figure 3 , it is a schematic flowchart of a model training method provided for some embodiments of this application. Figure 3 In , the model training method may include the following steps:

[0043] Specifically, for the scenario where multiple model training devices execute the model training task in parallel, the multiple model training devices can be connected to the host computer. In the host computer, the model training task can be divided into multiple subtasks, and moreover, the host computer can send the multiple subtasks to different model training devices in the form of electrical signals. Each model training device can receive and execute the subtasks sent by the host computer. For example, in the host computer, the model training task can be divided into subtasks B1, B2, and B3 according to the neural network layers, where subtask B1 represents the training task of neural network layers L1 to L3, subtask B2 represents the training task of neural network layers L4 to L5, and subtask B3 represents the training task of neural network layers L6 to L7. In the form of electrical signals, the host computer can send subtask B1 to model training device M1, send subtask B2 to model training device M2, and send subtask B3 to model training device M3. After each model training device receives the model training task, the central processing unit or graphics processing unit in the model training device can extract the matrix multiplication operation from the model training task. It can be understood that since the model training tasks executed by different model training devices are different, the matrix multiplication operations extracted by each model training device can be different.

[0044] For the scenario where all model training tasks are executed by the same model training device. The central processing unit or graphics processing unit in the model training device can extract all the matrix multiplication operations involved in the model training process.

[0045] In the matrix multiplication operation, at least the first matrix and the second matrix can be included. The first matrix can be regarded as the matrix to be multiplied, and the second matrix can be regarded as the multiplying matrix. The multiplication of the first matrix and the second matrix can be regarded as performing a matrix transformation on the first matrix based on the second matrix to obtain the result matrix. The second matrix defines the linear transformation from the first matrix to the result matrix.

[0046] It should be noted that since the roles of the first matrix and the second matrix are mutual, in some other embodiments, the multiplication of the first matrix and the second matrix can also be regarded as performing a matrix transformation on the second matrix based on the first matrix to obtain the result matrix. The first matrix defines the linear transformation from the second matrix to the result matrix.

[0047] Step S302: Set the transformation parameters of the photonic arithmetic unit according to the electrical signals corresponding to the second matrix. The transformation parameters represent the matrix transformation to be performed on the first matrix in the matrix multiplication operation.

[0048] In this embodiment, the central processing unit or the graphics processing unit in the model training device can set the transformation parameters of the photonic arithmetic unit based on the extracted second matrix. The process of setting the transformation parameters can be understood as configuring the optical paths in the photonic arithmetic unit. Different optical paths in the photonic arithmetic unit can be used to transform different elements in the first matrix. According to the specific values of the elements in the second matrix, each optical path can have its corresponding configuration.

[0049] Step S303: Convert the electrical signals corresponding to the first matrix into first optical signals, and input the first optical signals into the photonic arithmetic unit to obtain the operation result of the matrix multiplication operation.

[0050] Specifically, the central processing unit or the graphics processing unit in the model training device can input the electrical signals corresponding to the first matrix into the optoelectronic conversion module. The optoelectronic conversion module can convert the electrical signals corresponding to the first matrix into first optical signals and input the first optical signals into the photonic arithmetic unit with the optical path configuration completed. The photonic arithmetic unit can convert the first optical signals according to the optical path configuration and output the converted second optical signals. The second optical signals can represent the transformation result of the first matrix.

[0051] Compared with the electronic arithmetic unit that relies on logic gate circuits to process bit by bit when performing matrix multiplication operations, which has the problem of high latency, the photonic arithmetic unit can complete matrix multiplication operations at the speed of light by utilizing the parallel propagation characteristics of light. The latency is only determined by the optical path length (nanosecond level) and there is no need to process data bit by bit. Therefore, the operation efficiency can be greatly improved, and thus the problem of low model training efficiency in the related art is solved. In addition, compared with the power consumption of the electronic arithmetic unit that surges with frequency and integration, since optical signals hardly generate heat during transmission, using a photonic arithmetic unit for matrix multiplication operations can greatly reduce power consumption.

[0052] Step S304: Perform model training based on the operation result of the matrix multiplication operation.

[0053] Specifically, the central processing unit or the graphics processing unit in the model training device can modify the model parameters or continue with subsequent operations based on the operation result of the matrix multiplication operation.

[0054] In summary, in the technical solutions of some embodiments of the present application, when receiving a model training task transmitted by an electrical signal, a matrix multiplication operation can be extracted from the model training task, and transformation parameters of the photonic arithmetic unit can be set according to the electrical signal corresponding to the second matrix in the matrix multiplication operation. In this way, after converting the electrical signal corresponding to the first matrix in the matrix multiplication operation into a first optical signal and inputting the first optical signal into the photonic arithmetic unit, the matrix multiplication operation can be completed. Compared with the electronic arithmetic unit that depends on bit-by-bit processing by logic gates when performing matrix multiplication operations and has a high latency problem, the photonic arithmetic unit can complete matrix multiplication operations at the speed of light by utilizing the parallel propagation characteristics of light. The latency is only determined by the optical path length (nanosecond level) and there is no need to process data bit by bit. Therefore, the operation efficiency can be greatly improved, and the problem of low model training efficiency in the related art is solved. In addition, compared with the power consumption of the electronic arithmetic unit that surges with frequency and integration, since the optical signal hardly generates heat during transmission, using the photonic arithmetic unit for matrix multiplication operations can greatly reduce the power consumption.

[0055] The following further describes the solution of the present application.

[0056] In some embodiments, considering that electronic computing has advantages such as mature technology, low cost, high integration, and good flexibility, and using an electronic arithmetic unit to perform non-matrix multiplication operations has a relatively small impact on the operation efficiency and model training efficiency. Therefore, the operations in the model training task can be performed based on a combination of optical computing and electronic computing. Specifically, in these embodiments, after receiving a model training task transmitted by an electrical signal, the model training task can be performed according to the following steps: 1) Extract matrix multiplication operations and non-matrix multiplication operations from the model training task. Among them, the non-matrix multiplication operation can include one or more non-matrix terms and the target operation type for the non-matrix terms. The non-matrix term refers to constants, variables, etc. represented in a non-matrix form. The target operation type can include, but is not limited to, addition operation, subtraction operation, division operation, multiplication operation, statistical operation, conditional operation, etc.

[0057] 2) Based on the above steps S301 to S304, use the photonic arithmetic unit to perform the matrix multiplication operation to obtain the matrix multiplication operation result represented by the optical signal.

[0058] 3) Input the electrical signals corresponding to the non-matrix terms in the non-matrix multiplication operation into an electronic arithmetic unit that supports the target operation type, and obtain the operation result of the non-matrix multiplication operation. Specifically, in a central processing unit and a graphics processing unit, electronic arithmetic units that support different operation types can be integrated, such as an electronic arithmetic unit that supports addition operations, an electronic arithmetic unit that supports subtraction operations, etc. Based on the target operation type in the non-matrix multiplication operation, after inputting the electrical signals corresponding to the non-matrix terms into the electronic arithmetic unit that supports the target operation type, the electronic arithmetic unit can output the non-matrix multiplication operation result represented by the electrical signals.

[0059] 4) Perform model training based on the matrix multiplication operation result and the non-matrix multiplication operation result.

[0060] In the above embodiments, using a photonic arithmetic unit to perform matrix multiplication operations can improve the operation efficiency and model training efficiency. Using an electronic arithmetic unit to perform non-matrix multiplication operations can improve the operation stability and reduce the operation cost. By combining optical computing and electronic computing, the advantages of optical computing and electronic computing can be fully utilized, so that while improving the model training efficiency, the cost of model training can be reduced, and the stability and flexibility of model training can be improved.

[0061] Further, in some embodiments, the non-matrix multiplication operation further includes matrix terms that perform non-multiplication operations, such as matrix terms that perform addition operations or subtraction operations. For the operations of these matrix terms, either optical computing or electronic computing can be used, and this application does not limit this.

[0062] In some embodiments, there may be a dependency relationship between the matrix multiplication operation and the non-matrix multiplication operation. The so-called dependency relationship means that the execution of the non-matrix multiplication operation depends on the operation result of the matrix multiplication operation, or the execution of the matrix multiplication operation depends on the operation result of the non-matrix multiplication operation. For example, in the above expressions (5) and (6), it is necessary to perform the non-matrix multiplication operation in expression (5) based on the matrix multiplication operation result of expression (6). .

[0063] In view of this, when extracting the matrix multiplication operation and the non-matrix multiplication operation from the model training task, the dependency relationship between the matrix multiplication operation and the non-matrix multiplication operation can be analyzed, and based on the dependency relationship, the execution order of the matrix multiplication operation and the non-matrix multiplication operation can be determined. Specifically, if the non-matrix multiplication operation depends on the operation result of the matrix multiplication operation, then the non-matrix multiplication operation is performed after the matrix multiplication operation. If the matrix multiplication operation depends on the operation result of the non-matrix multiplication operation, then the matrix multiplication operation is performed after the non-matrix multiplication operation. In this way, it can be ensured that each operation in the model training task can be executed in an orderly manner, and the problem of the operation task execution failure caused by not obtaining the operation result on which the operation task depends can be prevented.

[0064] In some embodiments, when the operation result of a non-matrix multiplication operation depends on the operation result of a matrix multiplication operation, it is required that the operation result of the matrix multiplication operation be represented by an electrical signal. However, in practice, the operation result of the matrix multiplication operation output by the photonic arithmetic unit is represented by an optical signal, which cannot meet the actual requirements. In view of this, after the first optical signal is input into the photonic arithmetic unit, the second optical signal output by the photonic arithmetic unit can also be converted into an electrical signal, and the converted electrical signal is used as the operation result of the matrix multiplication operation. In this way, the electrical signal representing the operation result of the matrix multiplication operation can be directly used as an input item for the non-matrix multiplication operation and participate in the non-matrix multiplication operation.

[0065] Of course, it can be understood that if there is a dependency relationship between matrix multiplication operations, it is not necessary to convert the second optical signal output by the photonic arithmetic unit into an electrical signal. For example, assuming that matrix multiplication operation a depends on the operation result of matrix multiplication operation b, after the photonic arithmetic unit outputs the operation result of matrix multiplication operation b in the form of a second optical signal, the second optical signal can be directly used as the input for matrix multiplication operation a. In this case, it is not necessary to convert the second optical signal output by the photonic arithmetic unit into an electrical signal. In this way, the signal conversion time can be reduced and the model training efficiency can be improved.

[0066] In some embodiments, each row of the first matrix can be used as a row vector respectively. The first matrix can include at least one row vector, and the first optical signal can include at least one set of sub-optical signals, and the at least one set of sub-optical signals corresponds one-to-one to the row vectors of the first matrix. The above-mentioned input of the first optical signal into the photonic arithmetic unit to obtain the operation result of the matrix multiplication operation includes: Sequentially inputting the sub-optical signals corresponding to each row vector into the photonic arithmetic unit to obtain the operation results of each row vector; Aggregating the operation results of each row vector to obtain the operation result of the matrix multiplication operation.

[0067] For example, assuming that the first matrix includes three row vectors a1, a2, and a3, the sub-optical signal corresponding to row vector a1 can be input into the photonic arithmetic unit to obtain the operation result q1 corresponding to row vector a1, the sub-optical signal corresponding to row vector a2 can be input into the photonic arithmetic unit to obtain the operation result q2 corresponding to row vector a2, and the sub-optical signal corresponding to row vector a3 can be input into the photonic arithmetic unit to obtain the operation result q3 corresponding to row vector a3. Finally, aggregating the operation results q1, q2, and q3 to obtain the operation result of the matrix multiplication operation.

[0068] In the above embodiments, determining the operation result of the matrix multiplication operation based on the row vector can reduce the design difficulty of the photonic arithmetic unit.

[0069] In some embodiments, a photonic arithmetic unit may be designed based on a Mach–Zehnder interferometer (MZI). A Mach–Zehnder interferometer is an optical silicon-based device based on the principle of optical interference, and its core function is to accurately control the amplitude and phase of an optical signal, thereby completing complex mathematical operations. Referring to Figure 4 , which is a schematic diagram of a module of a Mach–Zehnder interferometer provided in some embodiments of this application. Figure 4 In [the figure], the Mach–Zehnder interferometer includes a first phase-shift arm 41, a second phase-shift arm 42, an input coupler 43, and an output coupler 44. Among them, the phase-shift arm is the optical path. The first phase-shift arm 41 includes one or more phase modulators 45. The second phase-shift arm 42, as a reference arm, may not include a phase modulator 45. The input coupler 43 can be used as a beam splitter for optical signals, and the output coupler 44 can be used as a beam combiner for optical signals. Both the input coupler 43 and the output coupler 44 can be referred to as 2×2 couplers. Based on the above structure, the Mach–Zehnder interferometer allows adjustment of the optical path difference to achieve interference of different phases. Specifically, when the input optical signal passes through the Mach–Zehnder interferometer, it will be split into two beams by the input coupler 43, and the two optical signals are respectively transmitted along the first phase-shift arm 41 and the second phase-shift arm 42. During the transmission process, the phase modulator 45 in the first phase-shift arm 41 will change the phase of the optical signal based on the configuration. After passing through the output coupler 44, the two beams of light will recombine spatially to form a new optical signal, which can be considered a linear combination of the input optical signal. Based on the above principle, the Mach–Zehnder interferometer can perform matrix operations. Specifically, the 2nd-order equivalent matrix of a Mach–Zehnder interferometer unit with two-port input and two-port output can be as shown in expression (7): (7) Where , represent the input optical signal, and can represent the phase changes of the input optical signal , . , represent the output optical signal.

[0070] Figure 4 The Mach–Zehnder interferometer in [the figure] can be used as a basic Mach–Zehnder interferometer unit. By topologically cascading Mach–Zehnder interferometer units, unitary matrices of any scale can be realized. For example, assuming that each rectangle in Figure 5 and Figure 6 is used as a Mach–Zehnder interferometer unit and arranged in a triangular array as shown in Figure 5 or Figure 6In the shown rectangular array arrangement, by topologically cascading Mach-Zehnder interferometer units, unitary matrices of any scale can be achieved.

[0071] Based on the above principle, with reference to Figure 7 , the schematic diagram of the internal module of the photon arithmetic unit provided by some embodiments of this application. Figure 7 In [reference], the photon arithmetic unit includes a first network, a second network, and an attenuator array. Among them, the first network represents the left unitary matrix obtained after performing singular value decomposition on the matrix, the second network represents the complex conjugate transpose of the right unitary matrix obtained after performing singular value decomposition on the matrix, and the attenuator array represents the rectangular diagonal matrix obtained after performing singular value decomposition on the matrix. Since any scale of unitary matrix can be achieved by topologically cascading Mach-Zehnder interferometer units, the above first network, second network, and attenuator array can be implemented by topologically cascading Mach-Zehnder interferometer units.

[0072] Based on the first network, second network, and attenuator array inside the photon arithmetic unit, setting the transformation parameters of the photon arithmetic unit according to the electrical signal corresponding to the second matrix may include: Performing singular value decomposition on the second matrix to obtain the left unitary matrix, the rectangular diagonal matrix, and the complex conjugate transpose of the right unitary matrix; Setting the transformation parameters of the photon arithmetic unit according to the electrical signal corresponding to the left unitary matrix, the electrical signal corresponding to the rectangular diagonal matrix, and the electrical signal corresponding to the complex conjugate transpose of the right unitary matrix.

[0073] In this way, it is equivalent to implementing the second matrix inside the photon arithmetic unit.

[0074] In the above embodiments, by topologically cascading Mach-Zehnder interferometer units to implement the second matrix inside the photon arithmetic unit, the feasibility of optical computing is enhanced.

[0075] In some embodiments, the first network and the second network respectively include multiple optical paths for transmitting the first optical signal, and the transformation parameters of the photon arithmetic unit include the attenuation coefficients of the attenuator array and the phase differences of different optical paths in the first network and the second network; The above setting of the transformation parameters of the photon arithmetic unit according to the electrical signal corresponding to the left unitary matrix, the electrical signal corresponding to the rectangular diagonal matrix, and the electrical signal corresponding to the complex conjugate transpose of the right unitary matrix includes: Setting the phase differences of different optical paths in the first network according to the electrical signal corresponding to the left unitary matrix; Setting the phase differences of different optical paths in the second network according to the electrical signal corresponding to the right unitary matrix; Setting the attenuation coefficients of the attenuator array according to the electrical signal corresponding to the rectangular diagonal matrix.

[0076] Thus, the related description of the model training method of this application is completed.

[0077] Corresponding to the model training method, this application also provides a model training device. Referring to Figure 8 , it is a schematic block diagram of the model training device 800 provided by some embodiments of this application. Figure 8 In [the figure], the model training device 800 includes a central processing unit 81, an optoelectronic conversion module 82, a graphics processing unit 83, and an optical integrated circuit 84. The graphics processing unit 83 includes an electronic arithmetic unit 831, and the optical integrated circuit 84 includes a photonic arithmetic unit 841.

[0078] One of the central processing unit 81 and the graphics processing unit 83 is used to receive a model training task transmitted using an electrical signal, extract a matrix multiplication operation including a first matrix and a second matrix from the model training task, input the electrical signal corresponding to the first matrix into the optoelectronic conversion module 82, and set the transformation parameters of the photonic arithmetic unit 841 according to the electrical signal corresponding to the second matrix. The transformation parameters represent the matrix transformation to be performed on the first matrix in the matrix multiplication operation; The optoelectronic conversion module 82 is used to convert the electrical signal corresponding to the first matrix into a first optical signal and input the first optical signal into the photonic arithmetic unit 841; The photonic arithmetic unit 841 is used to transform the first optical signal according to the transformation parameters to obtain the operation result of the matrix multiplication operation; One of the central processing unit 81 and the graphics processing unit 83 performs model training according to the operation result of the matrix multiplication operation.

[0079] In some embodiments, one of the central processing unit 81 and the graphics processing unit 83 is further used to perform the following operations: Extract a non-matrix multiplication operation from the model training task. The non-matrix multiplication operation includes one or more non-matrix terms and a target operation type for the non-matrix terms; Input the electrical signal corresponding to the non-matrix term into the electronic arithmetic unit 831 that supports the target operation type to obtain the operation result of the non-matrix multiplication operation; Perform model training according to the operation result of the non-matrix multiplication operation.

[0080] In some embodiments, the first matrix includes at least one row vector, the first optical signal includes at least one set of sub-optical signals, and at least one set of sub-optical signals corresponds one-to-one with the row vectors of the first matrix; The optoelectronic conversion module 82 is specifically used to sequentially input the sub-optical signals corresponding to each row vector into the photonic arithmetic unit 841; The photonic arithmetic unit 841 is specifically used to transform the sub-optical signals corresponding to each row vector respectively according to the transformation parameters to obtain the operation result of each row vector; One of the central processing unit 81 and the graphics processing unit 83 is specifically configured to aggregate the operation results of each row vector to obtain the operation result of the matrix multiplication operation.

[0081] In some embodiments, one of the central processing unit 81 and the graphics processing unit 83 is further configured to perform the following operations: Perform singular value decomposition on the second matrix to obtain the left unitary matrix, the rectangular diagonal matrix, and the complex conjugate transpose of the right unitary matrix; Set the transformation parameters of the photonic arithmetic unit 841 according to the electrical signals corresponding to the left unitary matrix, the electrical signals corresponding to the rectangular diagonal matrix, and the electrical signals corresponding to the complex conjugate transpose of the right unitary matrix.

[0082] In some embodiments, the photonic arithmetic unit 841 includes a first network, a second network, and an attenuator array. The first network and the second network respectively include multiple optical paths for transmitting the first optical signal. The transformation parameters of the photonic arithmetic unit 841 include the attenuation coefficients of the attenuator array and the phase differences of different optical paths in the first network and the second network; One of the central processing unit 81 and the graphics processing unit 83 is further configured to perform the following operations: Set the phase differences of different optical paths in the first network according to the electrical signals corresponding to the left unitary matrix; Set the phase differences of different optical paths in the second network according to the electrical signals corresponding to the right unitary matrix; Set the attenuation coefficients of the attenuator array according to the electrical signals corresponding to the rectangular diagonal matrix.

[0083] Regarding the working principle and beneficial effects of the model training device, reference may be made to the relevant descriptions of the model training method, which will not be elaborated here.

[0084] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation manner.

[0085] With reference to Figure 9 , an embodiment of the present application further provides an electronic device, including a memory 10 and a processor 20. The memory 10 stores a computer program, and the processor 20 is configured to run the computer program to execute the steps in any one of the embodiments of the above state data saving method.

[0086] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, wherein the computer program is configured to execute the steps in any one of the embodiments of the above state data saving method when running.

[0087] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media that can store computer programs such as USB flash drives, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disks, magnetic disks, or optical discs.

[0088] An embodiment of the present application also provides a computer program product. The above computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above embodiments of the state data saving method.

[0089] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps in any of the above embodiments of the state data saving method.

[0090] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0091] The above has introduced in detail a model training method, device, equipment, and storage medium provided by the present application. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A model training method, characterized in that: The method comprises: Receiving a model training task transmitted using an electrical signal, and extracting a matrix multiplication operation from the model training task, the matrix multiplication operation including a first matrix and a second matrix; According to the electrical signal corresponding to the second matrix, setting a transformation parameter of the photon operator, wherein the transformation parameter represents a matrix transformation to be performed on the first matrix in the matrix multiplication operation; Converting the electrical signal corresponding to the first matrix into a first optical signal, and inputting the first optical signal into the photon operator to obtain the operation result of the matrix multiplication operation; Model training is performed based on the calculation result of the matrix multiplication operation.

2. The method according to claim 1, characterized in that The method further comprises: Extracting a non-matrix multiplication operation from the model training task, the non-matrix multiplication operation comprising one or more non-matrix terms and a target operation type for the non-matrix terms; Inputting the electrical signal corresponding to the non-matrix item into an electronic operator supporting the target operation type to obtain the operation result of the non-matrix multiplication operation; Model training is performed based on the operation result of the non-matrix multiplication operation.

3. The method according to claim 2, characterized in that When extracting the matrix multiplication operation and the non-matrix multiplication operation from the model training task, the method further includes: Analyzing a dependency relationship between the matrix multiplication operation and the non-matrix multiplication operation; Based on the dependency, the execution order of the matrix multiplication operation and the non-matrix multiplication operation is determined, wherein if the non-matrix multiplication operation depends on the operation result of the matrix multiplication operation, the non-matrix multiplication operation is performed after the matrix multiplication operation; if the matrix multiplication operation depends on the operation result of the non-matrix multiplication operation, the matrix multiplication operation is performed after the non-matrix multiplication operation.

4. The method according to claim 1 or 2, characterized in that: The first matrix includes at least one row vector, the first optical signal includes at least one group of sub-optical signals, and the at least one group of sub-optical signals corresponds to the row vectors of the first matrix in a one-to-one manner; The step of inputting the first optical signal into the photon operator to obtain the result of the matrix multiplication operation includes: Inputting the sub-light signals corresponding to each row vector into the photon operator in sequence to obtain the operation results of each row vector; The operation results of each row vector are summarized to obtain the operation result of the matrix multiplication operation.

5. The method according to claim 1 or 2, characterized in that: The step of setting the transformation parameters of the photon operator according to the electrical signal corresponding to the second matrix includes: Performing singular value decomposition on the second matrix to obtain a left unitary matrix, a rectangular diagonal matrix, and a complex conjugate transpose of a right unitary matrix; The transformation parameters of the photon operator are set according to the electrical signal corresponding to the left unitary matrix, the electrical signal corresponding to the rectangular diagonal matrix and the electrical signal corresponding to the complex conjugate transpose of the right unitary matrix.

6. The method according to claim 5, characterized in that The photon operator comprises a first network, a second network and an attenuator array, wherein the first network and the second network respectively comprise a plurality of optical paths for transmitting the first optical signal, and the transformation parameters of the photon operator comprise an attenuation coefficient of the attenuator array and a phase difference between different optical paths in the first network and the second network; The step of setting the transformation parameters of the photon operator according to the electrical signal corresponding to the left unitary matrix, the electrical signal corresponding to the rectangular diagonal matrix, and the electrical signal corresponding to the complex conjugate transpose of the right unitary matrix comprises: According to the electrical signal corresponding to the left unitary matrix, setting the phase difference of different optical paths in the first network; According to the electrical signal corresponding to the right unitary matrix, setting the phase difference of different optical paths in the second network; The attenuation coefficient of the attenuator array is set according to the electrical signal corresponding to the rectangular diagonal matrix.

7. The method according to claim 1 or 2, characterized in that: The photon operator is used to convert the first optical signal according to the transformation parameter, and output a second optical signal obtained by the conversion; The step of inputting the first optical signal into the photon operator to obtain the result of the matrix multiplication operation includes: The second optical signal output by the photon operator is converted into an electrical signal, and the converted electrical signal is used as the operation result of the matrix multiplication operation.

8. A model training device, characterized in that: The model training device includes a central processing unit, a photoelectric conversion module, a graphics processor and an optical integrated circuit, wherein the graphics processor includes an electronic operator, and the optical integrated circuit includes a photon operator, wherein: One of the central processing unit and the graphics processing unit is used to receive a model training task transmitted using an electrical signal, and extract a matrix multiplication operation including a first matrix and a second matrix from the model training task, and input the electrical signal corresponding to the first matrix into the photoelectric conversion module, and set the transformation parameters of the photon operator according to the electrical signal corresponding to the second matrix, wherein the transformation parameters represent the matrix transformation to be performed on the first matrix in the matrix multiplication operation; The photoelectric conversion module is used to convert the electrical signal corresponding to the first matrix into a first optical signal, and input the first optical signal into the photon operator; The photon operator is used to transform the first optical signal according to the transformation parameter to obtain the operation result of the matrix multiplication operation; One of the central processing unit and the graphics processing unit performs model training based on the operation result of the matrix multiplication operation.

9. The device according to claim 8, characterized in that One of the central processing unit and the graphics processing unit is further configured to perform the following operation: Extracting a non-matrix multiplication operation from the model training task, the non-matrix multiplication operation comprising one or more non-matrix terms and a target operation type for the non-matrix terms; Inputting the electrical signal corresponding to the non-matrix item into an electronic operator supporting the target operation type to obtain the operation result of the non-matrix multiplication operation; Model training is performed based on the operation result of the non-matrix multiplication operation.

10. The device according to claim 8 or 9, characterized in that The first matrix includes at least one row vector, the first optical signal includes at least one group of sub-optical signals, and the at least one group of sub-optical signals corresponds to the row vectors of the first matrix in a one-to-one manner; The photoelectric conversion module is specifically used to input the sub-light signals corresponding to each row vector into the photon operator in sequence; The photon operator is specifically used to transform the sub-light signals corresponding to each row vector according to the transformation parameters to obtain the operation results of each row vector; One of the central processing unit and the graphics processing unit is specifically used to summarize the operation results of each row vector to obtain the operation result of the matrix multiplication operation.

11. The device according to claim 8 or 9, characterized in that One of the central processing unit and the graphics processing unit is further configured to perform the following operation: Performing singular value decomposition on the second matrix to obtain a left unitary matrix, a rectangular diagonal matrix, and a complex conjugate transpose of a right unitary matrix; The transformation parameters of the photon operator are set according to the electrical signal corresponding to the left unitary matrix, the electrical signal corresponding to the rectangular diagonal matrix and the electrical signal corresponding to the complex conjugate transpose of the right unitary matrix.

12. The device according to claim 11, characterized in that The photon operator comprises a first network, a second network and an attenuator array, wherein the first network and the second network respectively comprise a plurality of optical paths for transmitting the first optical signal, and the transformation parameters of the photon operator comprise an attenuation coefficient of the attenuator array and a phase difference between different optical paths in the first network and the second network; One of the central processing unit and the graphics processing unit is further configured to perform the following operation: According to the electrical signal corresponding to the left unitary matrix, setting the phase difference of different optical paths in the first network; According to the electrical signal corresponding to the right unitary matrix, setting the phase difference of different optical paths in the second network; The attenuation coefficient of the attenuator array is set according to the electrical signal corresponding to the rectangular diagonal matrix.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

14. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images