Intelligent light calculation large model training method and system

Abstract

The invention relates to the technical field of optical computing, in particular to an intelligent optical computing large model training method and system. The method comprises the following steps: acquiring a training data set, and inputting data in the training data set into a preset large model to obtain a prediction result; based on the prediction result and the real result, determining a corresponding target loss value through a target loss function; performing reverse gradient propagation based on the target loss value, and determining a first loss gradient corresponding to the multiple perceptron layers; determining a second loss gradient of each optical module layer based on the first loss gradient; and on the basis of the second loss gradient of each optical module layer, updating parameters in each optical module layer, and repeating the steps until the preset large model converges to obtain a target large model. The optical module layer is integrated into the multiple perceptron layers, so that efficient acceleration of optical matrix operation is achieved through the optical module layer, calculation delay and power consumption are reduced, and the training efficiency of a large model is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of optical computing technology, and in particular to a method and system for training a large model of intelligent optical computing. Background Art

[0002] With the development of deep learning and large models, the complexity and scale of computing requirements for the model training process of large models are also increasing. However, the existing electronic computing architecture faces high power consumption and computing delays when processing large-scale parallel computing tasks, and it is difficult to effectively cope with the increasingly stringent demands of large models for computing power and power consumption. Among them, optical computing uses photons for information transmission and processing, and has the advantages of ultra-high speed and low energy consumption. Based on this, optical computing technology that uses photons instead of electrons as computing carriers is regarded as the key to breaking the existing computing bottleneck. Summary of the invention

[0003] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0004] To this end, the first purpose of the present invention is to propose a large-model training method for intelligent optical computing. By integrating the optical module layer into the multi-layer perception machine layer, the optical module layer can be used to achieve efficient acceleration of optical matrix operations, reduce calculation delays and power consumption, and improve the training efficiency of large models.

[0005] The second objective of the present disclosure is to propose an intelligent optical computing large model training system.

[0006] To achieve the above-mentioned purpose, the first aspect of the present disclosure provides a method for training a large model of intelligent optical computing, wherein the large model includes a multi-layer perceptron layer, and the multi-layer perceptron layer includes at least one optical module layer. The method includes: Obtaining a training data set, and inputting data in the training data set into a preset large model to obtain a prediction result; Based on the predicted result and the actual result, a corresponding target loss value is determined by a target loss function; Perform reverse gradient propagation based on the target loss value to determine a first loss gradient corresponding to the multi-layer perceptron layer; Based on the first loss gradient, determining a second loss gradient for each optical module layer; Based on the second loss gradient of each optical module layer, the parameters in each optical module layer are updated, and the above steps are repeated until the preset large model converges to obtain the target large model.

[0007] Optionally, inputting the data in the training data set into a preset large model to obtain a prediction result includes: Acquire input data of the multi-layer perceptron layer, and input the input data into each optical module layer in parallel to obtain first output data of each optical module layer; Merging the first output data of each optical module layer to obtain the second output data of the multi-layer perceptron layer; The second output data is input into the next network layer of the multi-layer perceptron layer, and the output result of the last network layer in the large model is determined as the prediction result.

[0008] Optionally, each optical module layer includes at least one network sublayer; and determining the second loss gradient of each optical module layer based on the first loss gradient includes: Based on the first loss gradient, determining a third loss gradient for each network sublayer in each optical module layer; The third loss gradient of all network sub-layers of each optical module layer is determined as the second loss gradient of each optical module layer.

[0009] Optionally, updating the parameters in each optical module layer based on the second loss gradient of each optical module layer includes: updating the parameters of each network sublayer based on the third loss gradient of each network sublayer in each optical module layer.

[0010] Optionally, determining a third loss gradient of each network sublayer in each optical module layer based on the first loss gradient includes: Based on the first loss gradient, differentiating the first loss gradient according to the chain rule to determine a fourth loss gradient for each network sublayer in each optical module layer; The fourth loss gradient of each network sublayer is processed according to Lorentz reciprocity to obtain the third loss gradient of each network sublayer in each optical module layer.

[0011] To achieve the above-mentioned purpose, the second aspect of the present disclosure proposes an intelligent optical computing large model training system, wherein the large model includes a multi-layer perceptron layer, the multi-layer perceptron layer includes at least one optical module layer, and the system includes: A processing module is used to obtain a training data set and input the data in the training data set into a preset large model to obtain a prediction result; A first determination module, configured to determine a corresponding target loss value through a target loss function based on the predicted result and the actual result; A second determination module is used to perform reverse gradient propagation based on the target loss value to determine a first loss gradient corresponding to the multi-layer perceptron layer; A third determination module, configured to determine a second loss gradient of each optical module layer based on the first loss gradient; An updating module is used to update the parameters in each optical module layer based on the second loss gradient of each optical module layer, and repeat the above steps until the preset large model converges to obtain a target large model.

[0012] Another object of the present invention is to provide an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the methods described in the above two aspects.

[0013] Another object of the present invention is to provide a computer storage medium, wherein the computer storage medium stores computer executable instructions; after the computer executable instructions are executed by a processor, the computer executes the methods described in the above two aspects.

[0014] In summary, the intelligent optical computing large model training method and system provided by the present invention integrates the optical module layer into the multi-layer perception machine layer, so that the optical module layer can be used to achieve efficient acceleration of optical matrix operations, reduce computing delays and power consumption, and improve the training efficiency of large models.

[0015] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which: Figure 1 A flowchart of a large model training method for intelligent optical computing provided by an embodiment of the present disclosure; Figure 2 A schematic diagram of the structure of an intelligent optical computing large model training system provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0018] The present disclosure is described in detail below with reference to specific embodiments.

[0019] Figure 1 The following is a flow chart of a method for training a large model of intelligent optical computing provided by an embodiment of the present disclosure. Figure 1 As shown, the method may include the following steps: Step S1, obtaining a training data set, and inputting the data in the training data set into a preset large model to obtain a prediction result; Step S2, based on the predicted result and the actual result, determine the corresponding target loss value through the target loss function; Step S3, performing reverse gradient propagation based on the target loss value to determine the first loss gradient corresponding to the multi-layer perceptron layer; Step S4, determining a second loss gradient of each optical module layer based on the first loss gradient; Step S5, based on the second loss gradient of each optical module layer, update the parameters in each optical module layer, repeat the above steps until the preset large model converges to obtain the target large model.

[0020] In one embodiment of the present disclosure, the above-mentioned large model can be applied to a variety of scenarios, such as intelligent question-answering scenarios.

[0021] In one embodiment of the present disclosure, the above-mentioned large model may include a multi-layer perceptron layer, wherein one or more layers in the large model may be a multi-layer perceptron layer, and the embodiment of the present disclosure does not limit this.

[0022] And, in one embodiment of the present disclosure, the above-mentioned multi-layer perceptron is a trainable module, that is, the parameters of the multi-layer perceptron layer are completely reconfigurable. Among them, since optical computing has the characteristics of high parallelism and high throughput, the high-dimensional multi-layer perceptron can be divided into multiple low-dimensional multi-layer perceptron structures implemented in parallel, that is, the multi-layer perceptron layer is divided into at least one optical module layer, so as to utilize the high parallel computing characteristics of optical computing, and highly parallel online training can be synchronously implemented on each optical module layer, thereby improving the training speed of the multi-layer perception.

[0023] In one embodiment of the present disclosure, for each layer of the multilayer perceptron, a full forward training method can be used for training. For example, in one embodiment of the present disclosure, a multilayer perceptron with an input of N×N can be divided into a combination of N optical modules. Specifically, for a multilayer perceptron, its input is , the output is , and the input and output satisfy ,in, , transform it into a combination of N optical modules, that is, , is the input of the i-th optical module, is the output of the ith optical module, where , the i-th row and the i-th column are all 1, and the rest of the elements are 0. Therefore, the N optical modules after the above division can be completely equivalent to ordinary multi-layer perceptrons. At this time, the training of the multi-layer perceptron can be completed by training the N combined optical modules.

[0024] Furthermore, in one embodiment of the present disclosure, the method of inputting the data in the training data set into the preset large model to obtain the prediction result may include the following steps: Step S11, obtaining input data of the multi-layer perceptron layer, and inputting the input data into each optical module layer in parallel to obtain first output data of each optical module layer; Step S12, merging the first output data of each optical module layer to obtain the second output data of the multi-layer perceptron layer; Step S13, input the second output data into the next network layer of the multi-layer perceptron layer, and determine the output result of the last network layer in the large model as the prediction result.

[0025] In one embodiment of the present disclosure, the method for merging the first output data of each optical module layer can refer to the above .

[0026] And, in one embodiment of the present disclosure, each of the above optical module layers includes at least one network sublayer. For example, in one embodiment of the present disclosure, each of the above optical module layers may include an activation layer and a propagation layer.

[0027] Among them, in one embodiment of the present disclosure, the method of determining the second loss gradient of each optical module layer based on the first loss gradient may include: determining the third loss gradient of each network sublayer in each optical module layer based on the first loss gradient; and determining the third loss gradient of all network sublayers of each optical module layer as the second loss gradient of each optical module layer.

[0028] In one embodiment of the present disclosure, the forward propagation of the above large model during the training process can be expressed as:

[0029] in, is the position vector of the midpoint of the input surface, indicating the position of the point in three-dimensional space, is the position vector of the midpoint of the output surface, indicating the position of the point in three-dimensional space. For arrive The Green function is a transfer function. For at point The input value at the location, the output can be transmitted to the subsequent layer by the nonlinear activation layer. .

[0030] And, in one embodiment of the present disclosure, the loss function can be evaluated based on the prediction results, in, is the predicted result, and T is the actual result. In order to effectively evaluate the loss, the loss function L can be used to calculate the first loss gradient corresponding to the multi-layer perceptron layer for:

[0031] Further, in one embodiment of the present disclosure, after obtaining the first loss gradient through the above steps, the third loss gradient of each network sublayer in each optical module layer can be determined based on the first loss gradient. Wherein, in one embodiment of the present disclosure, the method for determining the third loss gradient of each network sublayer in each optical module layer based on the first loss gradient may include: based on the first loss gradient, differentiating the first loss gradient according to the chain rule to determine the fourth loss gradient of each network sublayer in each optical module layer; processing the fourth loss gradient of each network sublayer according to Lorentz reciprocity to obtain the third loss gradient of each network sublayer in each optical module layer.

[0032] In one embodiment of the present disclosure, the fourth loss gradient of each network sublayer in each optical module layer is determined by differentiating the first loss gradient according to the chain rule as follows:

[0033] And, in one embodiment of the present disclosure, the fourth loss gradient of each network sublayer is processed according to the Lorentz reciprocity, and the third loss gradient of each network sublayer in each optical module layer is obtained as follows:

[0034] According to the Lorentz reciprocity, we have , so we can Substitute and get the third loss gradient of each network sublayer.

[0035] It should be noted that in one embodiment of the present disclosure, the back propagation complexity of the gradient algorithm may hinder the training of a general system. Based on this, in order to adapt to free space and integrated optical systems, the optical system can be trained by forward propagation. At this time, it is necessary to prove that data and error propagation can share the same forward path in the case of spatial symmetry.

[0036] Among them, in the mirror symmetric system ,in, yes The position vector of the corresponding three-dimensional point obtained by mirror symmetry in the system is It can be achieved and One-to-one correspondence. And, for each exist Make ,Will Substituting into the above forward propagation process, we get , which is equivalent to providing And propagate it forward to the output end.

[0037] And, in the rotationally symmetric system In the above example, it is assumed that the input and output of the system have an integer rotational symmetry relationship, that is, for any input ,exist , and vice versa for any point in the output ,exist ,make ,when Sometimes, there is , so that At this time, Substitution , and again get , which is equivalent to providing And propagate it forward to the output end. Among them, the equivalent relationship The required rotation angle is radians, and is an integer multiple of the symmetric rotation, that is , the corresponding sufficient condition is Is an even number.

[0038] Further, in one embodiment of the present disclosure, in order to verify the accuracy of spatial symmetry and complex field measurement, the spatial symmetric conjugate method is used, where the transfer function is , x is the input, B is the input position, M is the trainable parameter, and W is the light propagation matrix. At this time, we need to analyze the propagation process . Where, when the system is unitary and symmetric, Will satisfy ,in is the identity matrix, and * represents the conjugate operation. After two propagations, and taking the conjugate of the first output as the input of the second propagation, we get .

[0039] Among them, in one embodiment of the present disclosure, the amplitude of the original input can be restored through two propagations under the constraints of symmetry and unitarity in the above manner. At this time, when the original input can be recaptured at the second output, the symmetry of the system can be verified. And, unitarity is a strict constraint facilitated by the losslessness of the system. In more general cases, the system may not always be unitary. For example, light may be scattered laterally or blocked in the lateral direction. In the above case, the propagation matrix is ​​expressed as ,in is a unitary matrix, H is and The normalized difference between Expressed as ,make is the eigenvalue with the largest magnitude in H, and its L2 norm is , which indicates that the similarity of the output is determined by Based on this, even under non-unitary conditions, the similarity between the output and input of the two propagations can be used as a measure to verify the symmetry of the optical system.

[0040] From the above proof, it can be seen that the multilayer perceptron can achieve efficient and high-speed intelligent optical module layer training through a full-forward training method. Based on this, the large model can achieve a high-speed and energy-efficient training process through efficient intelligent optical module layer training based on a full-forward online training method, providing a new, efficient and green solution for future large model training.

[0041] Furthermore, in one embodiment of the present disclosure, after obtaining the third loss gradient of each network sublayer in each optical module layer through the above steps, the parameters in each network sublayer can be updated based on the third loss gradient of each network sublayer in each optical module layer, and the above steps can be repeated until the preset large model converges to obtain the target large model.

[0042] Further, in one embodiment of the present disclosure, after the target big model is obtained through the above steps, the target big model can be applied to the intelligent question-answering scenario. Specifically, in one embodiment of the present disclosure, after the target big model obtains the input question data, the target answer corresponding to the question data is obtained through the above target big model.

[0043] In the intelligent optical computing large model training method of the disclosed embodiment, by integrating the optical module layer into the multi-layer perception machine layer, the optical module layer can be used to achieve efficient acceleration of optical matrix operations, reduce computing delays and power consumption, and improve the training efficiency of the large model.

[0044] In order to implement the above embodiment, Figure 2The present disclosure also proposes an intelligent optical computing large model training system, the large model includes a multi-layer perceptron layer, and the multi-layer perceptron layer includes at least one optical module layer. Figure 2 As shown, the system includes: a processing module 201, a first determination module 202, a second determination module 203, a third determination module 204 and an updating module 205, wherein: Processing module 201, used to obtain a training data set, and input the data in the training data set into a preset large model to obtain a prediction result; A first determination module 202 is used to determine a corresponding target loss value through a target loss function based on the predicted result and the actual result; A second determination module 203 is used to perform reverse gradient propagation based on the target loss value to determine a first loss gradient corresponding to the multi-layer perceptron layer; A third determination module 204, configured to determine a second loss gradient of each optical module layer based on the first loss gradient; The updating module 205 is used to update the parameters in each optical module layer based on the second loss gradient of each optical module layer, and repeat the above steps until the preset large model converges to obtain the target large model.

[0045] In the embodiment of the present disclosure, the processing module 201 is specifically used for: Obtain input data of the multi-layer perceptron layer, and input the input data into each optical module layer in parallel to obtain first output data of each optical module layer; Merging the first output data of each optical module layer to obtain the second output data of the multi-layer perceptron layer; The second output data is input into the next network layer of the multi-layer perceptron layer, and the output result of the last network layer in the large model is determined as the prediction result.

[0046] In the embodiment of the present disclosure, the third determining module 204 is specifically used to: Based on the first loss gradient, determining a third loss gradient for each network sublayer in each optical module layer; The third loss gradient of all network sub-layers of each optical module layer is determined as the second loss gradient of each optical module layer.

[0047] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this disclosure shall comply with the relevant laws and regulations and shall not violate public order and good morals.

[0048] It should be noted that personal information from users should be collected for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign the agreement / authorization including authorization of relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others who have access to personal information data comply with its privacy policy and procedures.

[0049] The present disclosure anticipates providing implementation schemes for users to selectively block the use or access of personal information data. That is, the present disclosure anticipates providing hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, risks can be minimized by limiting data collection and deleting the data. In addition, when applicable, such personal information is de-identified to protect the privacy of the user.

[0050] The acquisition, transmission, storage, use, and processing of data in the technical solution disclosed in this disclosure are in compliance with the relevant provisions of national laws and regulations.

[0051] It should be noted that in the embodiments of the present disclosure, certain software, components, models and other existing solutions in the industry may be mentioned, which should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0052] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they contradict each other.

[0053] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0054] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.

[0055] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0056] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0057] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0058] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0059] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present disclosure. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present disclosure.

Claims

1. A method for training a large model of intelligent optical computing, characterized in that: The large model includes a multi-layer perceptron layer, the multi-layer perceptron layer includes at least one optical module layer, and the method includes: Obtaining a training data set, and inputting data in the training data set into a preset large model to obtain a prediction result; Based on the predicted result and the actual result, a corresponding target loss value is determined by a target loss function; Perform reverse gradient propagation based on the target loss value to determine a first loss gradient corresponding to the multi-layer perceptron layer; Based on the first loss gradient, determining a second loss gradient for each optical module layer; Based on the second loss gradient of each optical module layer, the parameters in each optical module layer are updated, and the above steps are repeated until the preset large model converges to obtain the target large model.

2. The method according to claim 1, characterized in that: The step of inputting the data in the training data set into a preset large model to obtain a prediction result includes: Acquire input data of the multi-layer perceptron layer, and input the input data into each optical module layer in parallel to obtain first output data of each optical module layer; Merging the first output data of each optical module layer to obtain the second output data of the multi-layer perceptron layer; The second output data is input into the next network layer of the multi-layer perceptron layer, and the output result of the last network layer in the large model is determined as the prediction result.

3. The method according to claim 1, characterized in that Each optical module layer includes at least one network sublayer; and determining the second loss gradient of each optical module layer based on the first loss gradient includes: Based on the first loss gradient, determining a third loss gradient for each network sublayer in each optical module layer; The third loss gradient of all network sub-layers of each optical module layer is determined as the second loss gradient of each optical module layer.

4. The method according to claim 3, characterized in that: The updating of parameters in each optical module layer based on the second loss gradient of each optical module layer includes: updating parameters of each network sublayer based on the third loss gradient of each network sublayer in each optical module layer.

5. The method according to claim 3, characterized in that: The determining, based on the first loss gradient, a third loss gradient of each network sublayer in each optical module layer includes: Based on the first loss gradient, differentiating the first loss gradient according to the chain rule to determine a fourth loss gradient for each network sublayer in each optical module layer; The fourth loss gradient of each network sublayer is processed according to Lorentz reciprocity to obtain the third loss gradient of each network sublayer in each optical module layer.

6. An intelligent optical computing large model training system, characterized in that: The large model includes a multi-layer perceptron layer, the multi-layer perceptron layer includes at least one optical module layer, and the system includes: A processing module is used to obtain a training data set and input the data in the training data set into a preset large model to obtain a prediction result; A first determination module, configured to determine a corresponding target loss value through a target loss function based on the predicted result and the actual result; A second determination module is used to perform reverse gradient propagation based on the target loss value to determine a first loss gradient corresponding to the multi-layer perceptron layer; A third determination module, configured to determine a second loss gradient of each optical module layer based on the first loss gradient; An updating module is used to update the parameters in each optical module layer based on the second loss gradient of each optical module layer, and repeat the above steps until the preset large model converges to obtain a target large model.

7. The system according to claim 6, characterized in that The processing module is specifically used for: Acquire input data of the multi-layer perceptron layer, and input the input data into each optical module layer in parallel to obtain first output data of each optical module layer; Merging the first output data of each optical module layer to obtain the second output data of the multi-layer perceptron layer; The second output data is input into the next network layer of the multi-layer perceptron layer, and the output result of the last network layer in the large model is determined as the prediction result.

8. The system according to claim 6, characterized in that Each optical module layer includes at least one network sublayer; the third determination module is specifically used to: Based on the first loss gradient, determining a third loss gradient for each network sublayer in each optical module layer; The third loss gradient of all network sub-layers of each optical module layer is determined as the second loss gradient of each optical module layer.

9. An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 5.

10. A computer storage medium, wherein: The computer storage medium stores computer executable instructions; after the computer executable instructions are executed by the processor, the method according to any one of claims 1 to 5 can be implemented.

Images