Model training method, device, electronic device and storage medium

Abstract

The present application discloses a model training method, device, electronic device and storage medium. The method includes: broadcasting a training data set corresponding to a target model in a peer-to-peer computing network, so that multiple nodes in the peer-to-peer computing network negotiate and allocate the training data set, and determine the training data subsets to be trained for each node; training the target model based on the target training data subset allocated to the target node, and obtaining a first training result corresponding to the target model; broadcasting the first training result in the peer-to-peer computing network, receiving the second training result broadcasted by other nodes in the peer-to-peer computing network, and the second training result is obtained by training the target model based on the corresponding allocated training data subset by other nodes in the peer-to-peer computing network; updating the target model based on the first training result and / or the second training result to obtain the first update information corresponding to the target model, reducing the equipment cost required for model training, and improving training efficiency and scalability.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model training method, device, electronic device and storage medium. Background Art

[0002] Machine learning and deep learning model training systems mainly adopt a centralized architecture, concentrating all model training tasks on the server cluster in the data center.

[0003] Currently, machine learning model training still relies on dedicated computing resources at the data center level, resulting in high training costs and limited expansion. Using the computing resources of idle devices at the edge of the network for distributed collaborative training is an effective way to accelerate and expand training. However, distributed collaborative training generally has the problem of difficulty in mobilizing nodes to participate and effectively allocating training tasks. Summary of the invention

[0004] The embodiments of the present application provide a model training method, device, electronic device and storage medium, which not only avoids the problem that model training relies on the dedicated computing resources of the data center server, resulting in high training costs and limited expansion, but also gives full play to the advantages of the peer-to-peer computing network to improve the efficiency and scalability of model training. The above technical solutions are as follows:

[0005] In a first aspect, an embodiment of the present application provides a model training method, which is applied to a target node in a peer-to-peer computing network, and the method includes:

[0006] Broadcasting a training data set corresponding to the target model in the peer-to-peer computing network, so that multiple nodes in the peer-to-peer computing network negotiate and allocate the training data set, and determining that each of the multiple nodes corresponds to a subset of training data to be trained; the training data set includes multiple training data, and the training data subset includes at least one of the training data;

[0007] The target model is trained based on the target training data subset allocated to the target node to obtain a first training result corresponding to the target model;

[0008] Broadcasting the first training result in the peer-to-peer computing network, and receiving the second training result broadcast by other nodes in the peer-to-peer computing network; the second training result is obtained by training the target model by other nodes in the peer-to-peer computing network based on the corresponding allocated training data subset;

[0009] The target model is updated based on the first training result and / or the second training result to obtain first update information corresponding to the target model.

[0010] In a possible implementation, after updating the target model based on the first training result and / or the second training result to obtain first update information corresponding to the target model, the method further includes:

[0011] Broadcasting the first update information in the peer-to-peer computing network, and receiving the second update information corresponding to the target model broadcasted by other nodes in the peer-to-peer computing network; the second update information is obtained by updating the target model by other nodes in the peer-to-peer computing network based on the first training result broadcasted by the target node and / or the second training result obtained by corresponding training of other nodes in the peer-to-peer computing network;

[0012] The target model updated corresponding to the first update information is updated based on the first update information and the second update information to obtain target update information corresponding to the target model.

[0013] In a possible implementation, the target model is a sub-model of a specified model, and the sub-model is obtained by splitting the specified model by the server;

[0014] Before broadcasting the training data set corresponding to the target model in the peer computing network, the method further includes:

[0015] Receiving the target model and the training data set corresponding to the target model sent by the server;

[0016] Broadcasting the above target model in the above peer computing network;

[0017] After updating the target model updated by the first update information based on the first update information and the second update information to obtain the target update information corresponding to the target model, the method further includes:

[0018] The target update information corresponding to the target model is sent to the server, so that the server aggregates the target update information sent by the target nodes in the multiple peer computing networks to obtain the designated update information corresponding to the target designated model.

[0019] In a possible implementation, updating the target model based on the first training result and / or the second training result to obtain first update information corresponding to the target model includes:

[0020] Determine whether the current computing power of the target node is greater than the first computing power value;

[0021] If yes, updating the target model based on the first training result and the second training result to obtain first update information corresponding to the target model;

[0022] If not, then when the current computing power is greater than the second computing power value, the target model is updated based on the first training result to obtain first update information corresponding to the target model;

[0023] Among them, the above-mentioned second computing power value is less than the above-mentioned first computing power value.

[0024] In a possible implementation, after receiving the second training result broadcasted by other nodes in the peer computing network, and before updating the target model based on the first training result and / or the second training result and obtaining the first update information corresponding to the target model, the method further includes:

[0025] Based on the signature certificates corresponding to the other nodes that broadcast the second training result, the signature carried by the second training result is verified to obtain a signature verification result corresponding to the second training result;

[0026] If the signature verification result is passed, the step of updating the target model based on the first training result and / or the second training result to obtain the first update information corresponding to the target model is performed.

[0027] In one possible implementation, the above-mentioned peer computing network includes the above-mentioned multiple nodes; the above-mentioned multiple nodes have passed the signature verification based on the corresponding signature certificates; the communication protocol versions corresponding to the above-mentioned multiple nodes are the same; and the current computing powers corresponding to the above-mentioned multiple nodes are all greater than the third computing power value.

[0028] In a possible implementation, the target model is trained based on the target training data subset allocated to the target node to obtain a first training result corresponding to the target model, including:

[0029] Adjusting the parameter dimension of the target model to be trained based on the length of each training data in the target training data subset allocated to the target node;

[0030] The target model after adjusting the parameter dimension is trained based on the target training data subset allocated to the target node to obtain a first training result corresponding to the target model.

[0031] In a second aspect, an embodiment of the present application provides a model training device, which is applied to a target node in a peer-to-peer computing network, and the model training device includes:

[0032] A first broadcast module is used to broadcast a training data set corresponding to a target model in the above-mentioned peer computing network, so that a plurality of nodes in the above-mentioned peer computing network negotiate and allocate the above-mentioned training data set, and determine that each of the above-mentioned plurality of nodes corresponds to a subset of training data to be trained; the above-mentioned training data set includes a plurality of training data, and the above-mentioned training data subset includes at least one of the above-mentioned training data;

[0033] A training module, used for training the target model based on the target training data subset allocated to the target node to obtain a first training result corresponding to the target model;

[0034] A second broadcast module, used for broadcasting the first training result in the peer computing network;

[0035] A first receiving module is used to receive a second training result broadcasted by other nodes in the above-mentioned peer computing network; the above-mentioned second training result is obtained by other nodes in the above-mentioned peer computing network training the above-mentioned target model based on the corresponding allocated training data subset;

[0036] The first updating module is used to update the target model based on the first training result and / or the second training result to obtain first updating information corresponding to the target model.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory;

[0038] The processor is connected to the memory;

[0039] The above-mentioned memory is used to store executable program code;

[0040] The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the method steps provided by the first aspect of the embodiment of this specification or any possible implementation of the first aspect.

[0041] In a fourth aspect, an embodiment of the present specification provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the method steps provided by the first aspect of the embodiment of the present specification or any possible implementation of the first aspect.

[0042] The beneficial effects brought about by the technical solutions provided by some embodiments of the present application include at least:

[0043] In one or more embodiments of the present application, a training data set corresponding to the target model is broadcast in the above-mentioned peer computing network, so that multiple nodes in the above-mentioned peer computing network negotiate and allocate the above-mentioned training data set, and determine that each of the above-mentioned multiple nodes corresponds to a training data subset to be trained; the above-mentioned training data set includes multiple training data, and the above-mentioned training data subset includes at least one of the above-mentioned training data; the above-mentioned target model is trained based on the target training data subset allocated to the above-mentioned target node, and a first training result corresponding to the above-mentioned target model is obtained; the above-mentioned first training result is broadcast in the above-mentioned peer computing network, and a second training result broadcasted by other nodes in the above-mentioned peer computing network is received. The second training result is obtained by training the target model by other nodes in the peer-to-peer computing network based on the corresponding allocated training data subset; the target model is updated based on the first training result and / or the second training result to obtain the first update information corresponding to the target model, thereby utilizing the computing resources of idle edge devices (nodes) in the peer-to-peer computing network for distributed collaborative training, which does not require a central node or server to allocate training tasks, avoids the problem that model training relies on the dedicated computing resources of data center nodes or servers, resulting in high training costs and limited expansion, and gives full play to the advantages of the peer-to-peer computing network to improve the efficiency and scalability of model training.

[0044] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A schematic diagram of the architecture of a model training system provided for an exemplary embodiment of the present application;

[0047] Figure 2 A flowchart of a model training method provided for an exemplary embodiment of the present application;

[0048] Figure 3 A flowchart of another model training method provided for an exemplary embodiment of the present application;

[0049] Figure 4A schematic diagram of the architecture of another model training system provided for an exemplary embodiment of the present application;

[0050] Figure 5 A schematic diagram of the structure of a model training device provided by an exemplary embodiment of the present application;

[0051] Figure 6 A schematic structural diagram of an electronic device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0053] The terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0054] Please refer to the following Figure 1 , which is a schematic diagram of the architecture of a model training system provided by an exemplary embodiment of this specification. Figure 1 As shown, the model training system may include: a plurality of nodes 110. Among them:

[0055] The above multiple nodes 110 are all located in the peer-to-peer computing network, and the above multiple nodes 110 have equal status in the peer-to-peer computing network. Devices at the edge of the network can spontaneously join the peer-to-peer computing network, and the overall peer-to-peer computing network is decentralized. Models and data can be directly transmitted between nodes in the peer-to-peer computing network. The above nodes 110 can be, but are not limited to, mobile phones, tablet computers, laptops and other devices.

[0056] Specifically, the above-mentioned multiple nodes 110 have passed the signature verification based on the corresponding signature certificate, thereby preventing counterfeit computing power devices (nodes) from participating in the model training in the peer-to-peer computing power network; in order to ensure that the multiple nodes 110 in the peer-to-peer computing power network can realize efficient model and data transmission, the above-mentioned multiple nodes 110 have the same corresponding communication protocol version; the current computing power corresponding to each of the above-mentioned multiple nodes is greater than the third computing power value, thereby excluding nodes with no computing power or insufficient computing power, and only allowing devices (nodes) with sufficient computing power to join the peer-to-peer computing power network to participate in model training. That is, when a new device wants to join the peer-to-peer computing power network, the devices in the peer-to-peer computing power network will perform signature verification, communication protocol version verification and computing power verification on it. If all verifications are passed, the new device can be registered in the peer-to-peer computing power network as a node in the peer-to-peer computing power network.

[0057] Specifically, the target node 110 in the peer-to-peer computing network can broadcast a training data set corresponding to the target model in the peer-to-peer computing network, so that multiple nodes in the peer-to-peer computing network negotiate and allocate the training data set, determine that each of the multiple nodes 110 corresponds to a training data subset to be trained, and the above-mentioned training data set includes multiple training data, and the training data subset includes at least one training data; based on the target training data subset corresponding to the target node 110, the target model is trained to obtain a first training result corresponding to the target model; the first training result is broadcast in the peer-to-peer computing network, and the second training result broadcast by other nodes 110 in the peer-to-peer computing network is received, and the above-mentioned second training result is obtained by training the target model by other nodes 110 in the peer-to-peer computing network based on the corresponding allocated training data subset; based on the first training result and / or the second training result, the target model is updated to obtain first update information corresponding to the target model.

[0058] It can be understood that the target node 110 in the above-mentioned peer computing network can be but is not limited to any one of the multiple nodes 110 in the peer computing network, and can also be the node with the largest computing power among the above-mentioned multiple nodes 110. The embodiment of the present application is not limited to this.

[0059] The network may be a medium that provides a communication link between any two nodes 110, or may be the Internet including network devices and transmission media, but is not limited thereto. The transmission medium may be a wired link, such as but not limited to coaxial cable, optical fiber, and digital subscriber line (DSL), or a wireless link, such as but not limited to wireless fidelity (WIFI), Bluetooth, and mobile device networks.

[0060] Understandably, Figure 1The number of nodes 110 in the model training system shown is for example only. In a specific implementation, the model training system may include any number of nodes 110 greater than or equal to 2. The nodes 110 may be, but are not limited to, any device that can provide computing power. The device models corresponding to the multiple nodes 110 may be the same or different, and the embodiments of this specification do not specifically limit this.

[0061] Please refer to the following Figure 2 , which exemplarily shows a flow chart of a model training method provided in an embodiment of the present application. Figure 2 As shown, the model training method includes the following steps:

[0062] S201, broadcasting a training data set corresponding to a target model in a peer-to-peer computing network, so that multiple nodes in the peer-to-peer computing network negotiate and allocate the training data set, and determine that each of the multiple nodes corresponds to a subset of training data to be trained.

[0063] Specifically, the above-mentioned training data set includes multiple training data, and the above-mentioned training data subset includes at least one training data. After the target node in the peer-to-peer computing network receives the target model and the training data set corresponding to the target model sent by the server or manually input by the user through the network, it can broadcast the training data set corresponding to the above-mentioned target model in the peer-to-peer computing network. After other nodes in the peer-to-peer computing network receive the training data set broadcasted by the target node, they will negotiate with the target node to select at least one training data (training data subset) in the training data set that each of them wants to train. The training data in the training data subset to be trained corresponding to each of the multiple nodes in the above-mentioned peer-to-peer computing network do not overlap.

[0064] It can be understood that the union of the training data subsets to be trained corresponding to the multiple nodes is the training data set, and the intersection between any two training data subsets to be trained corresponding to the multiple nodes is an empty set. The number of training data in the training data subsets to be trained corresponding to the multiple nodes can be the same or different, and can be negotiated and allocated based on the current computing resources of each node and the actual model training situation, which is not limited in the embodiments of the present application.

[0065] It can be understood that in addition to broadcasting the training data set corresponding to the target model in the peer-to-peer computing network, the target node will also broadcast the target model to be trained. That is, the above-mentioned target model to be trained can be sent to each node in the peer-to-peer computing network in the form of broadcasting, so that each node in the subsequent peer-to-peer computing network can complete the training of the target model based on the corresponding negotiated allocated training data subset.

[0066] S202: Train the target model based on the target training data subset allocated to the target node to obtain a first training result corresponding to the target model.

[0067] Specifically, after the target node determines through negotiation the target training data subset to be trained, it can train the target model to be trained based on the target training data subset to obtain a first training result corresponding to the target model. The above-mentioned first training result may include but is not limited to including the difference values ​​of the parameters in the target model that need to be modified after the target model is trained with the target training data subset or the target parameters obtained after the parameters in the target model are trained.

[0068] Optionally, since the computing power required for training different parameter dimensions is different, after the target node determines the target training data subset to be trained through negotiation, the parameter dimension of the target model to be trained can be adjusted based on the length of each training data in the target training data subset allocated to the target node. For example, when each training data in the target training data subset is less than or equal to 100 characters, the parameter dimension of the target model to be trained can be adjusted to 125 characters, which is slightly larger than 100 characters, without selecting 1024 characters or 4096 characters with a larger difference. Then, the target model with adjusted parameter dimension is trained based on the target training data subset allocated to the target node to obtain the first training result corresponding to the target model, thereby dynamically adjusting the parameter dimension of the target model to be trained according to the length of the allocated training data, reducing the computing power resources required for each node to perform model training, and improving the efficiency and intelligence of model training.

[0069] Optionally, the number of training data in the target training data subset allocated to the above-mentioned target node can be updated based on the update of the number of nodes in the peer computing power network, that is, the number of training data that can be allocated to each node in the peer computing power network can be determined according to the total number of nodes in the peer computing power network. When the total number of nodes in the peer computing power network changes, the number of training data that can be allocated to each node will also change accordingly. This not only avoids the problem of model training interruption caused by temporary joining and exiting of nodes in the peer computing power network, affecting the model training speed, but also can flexibly adjust the system parameters of the peer computing power network according to the computing power of the nodes in the peer computing power network, further ensuring the efficiency of model training and improving the intelligence of model training.

[0070] S203: broadcast the first training result in the peer-to-peer computing network, and receive the second training result broadcast by other nodes in the peer-to-peer computing network.

[0071] Specifically, after the target node trains the target model based on the allocated target training data subset and obtains the first training result, the first training result can be broadcast in the peer-to-peer computing network to share the first training result obtained after the training with other nodes in the peer-to-peer computing network. The first training result can carry a corresponding target training data identifier, which is used to characterize the identity of the training data used in the training corresponding to the first training result. At the same time, the target node can also receive the second training result broadcast and shared by other nodes in the peer-to-peer computing network. The second training result is obtained by other nodes in the peer-to-peer computing network training the target model based on the corresponding allocated training data subset. The second training result can also carry a corresponding training data identifier, which is used to characterize the identity of the training data used in the training corresponding to the second training result.

[0072] It can be understood that the number of other nodes in the peer-to-peer computing network can be one or more, and the target node can be any node in the peer-to-peer computing network, which is not limited in this embodiment of the present application.

[0073] S204: Update the target model based on the first training result and / or the second training result to obtain first update information corresponding to the target model.

[0074] Specifically, after the target node receives the second training result broadcasted by other nodes in the peer computing network, it can directly update its own stored target model based on the first training result obtained by its own training and / or the second training result obtained by other nodes' training, and obtain the first update information corresponding to the target model. The above-mentioned first update information may include the updated target model or the target parameter of the updated target model, and may, but is not limited to, carry the version information of the updated target model, such as, but not limited to, the training data identifier involved in the update of the updated target model, that is, which training data are used in the training of the updated target model.

[0075] It can be understood that the number of the above-mentioned second training results can be 1 or more, which can specifically correspond to the actual number of other nodes in the peer-to-peer computing network except the target node, and the embodiment of the present application does not limit this. The number of second training results used when the target node updates the target model can be 0, 1, or more, etc., which can be selected based on the current computing power of the target node and the computing power required for each second training result to update the target model, and the embodiment of the present application does not limit this.

[0076] In an embodiment of the present application, a training data set corresponding to a target model is broadcasted in the above-mentioned peer computing network, so that multiple nodes in the above-mentioned peer computing network negotiate and allocate the above-mentioned training data set, and determine that each of the above-mentioned multiple nodes corresponds to a training data subset to be trained; the above-mentioned training data set includes multiple training data, and the above-mentioned training data subset includes at least one of the above-mentioned training data; the above-mentioned target model is trained based on the target training data subset allocated to the above-mentioned target node, and a first training result corresponding to the above-mentioned target model is obtained; the above-mentioned first training result is broadcasted in the above-mentioned peer computing network, and the first training result broadcasted by other nodes in the above-mentioned peer computing network is received. Second training result; the second training result is obtained by training the target model by other nodes in the peer-to-peer computing network based on the corresponding allocated training data subset; the target model is updated based on the first training result and / or the second training result to obtain the first update information corresponding to the target model, thereby utilizing the computing resources of idle edge devices (nodes) in the peer-to-peer computing network for distributed collaborative training, thereby avoiding the problem of high training costs and limited expansion due to the large training data set and the reliance of model training on dedicated computing resources of data center servers, and giving full play to the advantages of the peer-to-peer computing network to improve the efficiency and scalability of model training.

[0077] Optionally, in order to avoid the problem of model update failure caused by insufficient computing power of the target node, the above S204, updating the target model based on the first training result and / or the second training result, and obtaining the first update information corresponding to the target model may include: first determining whether the current computing power of the target node is greater than the first computing power value, and the above first computing power value may be any value greater than or equal to the total computing power required to update the target model based on the first training result and the second training result; if the current computing power of the target node is greater than the first computing power value, it means that the target node currently has sufficient computing power to aggregate the first training result and the second training result, and the target model can be directly updated based on the first training result and the second training result to obtain the first update information corresponding to the target model; if the current computing power of the target node is less than or equal to the first computing power value, it means that the current computing power of the target node is insufficient to aggregate the first training result and the second training result, and the target model can be updated based on the first training result when the current computing power is greater than the second computing power value, and the first update information corresponding to the target model is obtained, and the above second computing power value may be any value greater than or equal to the total computing power required to update the target model based on the first training result and less than the first computing power value.

[0078] Optionally, if the current computing power of the target node is less than or equal to the second computing power value, indicating that the current computing power of the target node is insufficient to update the target model, the target node can directly receive the second update information broadcast by other nodes after obtaining the first training result after broadcasting its training in the peer computing power network, after updating the target model based on the first training result and / or the second training result, and update the stored target model to the target model in the second update information (the target model updated by other nodes).

[0079] Optionally, in order to ensure the accuracy of the source of the second training result, after receiving the second training result broadcasted by other nodes in the peer-to-peer computing network, the target node updates the target model based on the first training result and / or the second training result, and before obtaining the first update information corresponding to the target model, it can also verify the signature carried by the second training result based on the signature certificate corresponding to the other nodes that broadcast the second training result, and obtain the signature verification result corresponding to the second training result; if the signature verification result is verified, it means that the second training result is indeed from the corresponding other nodes in the peer-to-peer computing network, and then the step of updating the target model based on the first training result and / or the second training result to obtain the first update information corresponding to the target model can be executed. The above-mentioned signature can be, but is not limited to, asymmetric encryption.

[0080] Optionally, after the target model is updated, the target node may also distribute corresponding reward points to other nodes based on the contributions of other nodes in this update process (for example, but not limited to the number of accurately trained training data in the second training result used for the update) to encourage nodes in the peer computing network to actively participate in model training and provide more efficient and accurate computing resources for model training.

[0081] Exemplarily, the model training method described in the embodiments of the present application can be applied to the training of any model such as a recognition model, a large language model, etc., but is not limited to the large language model. The large language model refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. The large language model can handle a variety of natural language tasks, such as text classification, question answering, dialogue, etc. The target model is, for example, but not limited to, a text classification model, and the training data set may include multiple training texts of known actual categories.

[0082] Please refer to the following Figure 3 , which exemplarily shows a flow chart of another model training method provided in an embodiment of the present application. Figure 3 As shown, the model training method includes the following steps:

[0083] S301, broadcasting a training data set corresponding to a target model in a peer-to-peer computing network, so that multiple nodes in the peer-to-peer computing network negotiate and allocate the training data set, and determine that each of the multiple nodes corresponds to a subset of training data to be trained.

[0084] Specifically, S301 is consistent with S201 and will not be described in detail here.

[0085] S302: Train the target model based on the target training data subset allocated to the target node to obtain a first training result corresponding to the target model.

[0086] Specifically, S302 is consistent with S202 and will not be repeated here.

[0087] S303: broadcast the first training result in the peer-to-peer computing network, and receive the second training result broadcast by other nodes in the peer-to-peer computing network.

[0088] Specifically, S303 is consistent with S203 and will not be described in detail here.

[0089] S304: Update the target model based on the first training result and / or the second training result to obtain first update information corresponding to the target model.

[0090] Specifically, S304 is consistent with S204 and will not be described in detail here.

[0091] S305, broadcasting first update information in the peer-to-peer computing network, and receiving second update information corresponding to the target model broadcasted by other nodes in the peer-to-peer computing network.

[0092] Specifically, after each node in the peer-to-peer computing network updates its own stored target model, the corresponding update information will be broadcast and shared in the peer-to-peer computing network. The above-mentioned second update information is obtained by updating the target model by other nodes in the peer-to-peer computing network based on the first training result broadcast by the target node and / or the second training result obtained by the corresponding training of other nodes in the peer-to-peer computing network. The above-mentioned second update information may include the target model updated by other nodes or the target parameters of the updated target model, and may, but is not limited to, carry the version information of the updated target model, such as, but not limited to, including the training data identifier involved in the corresponding update of the updated target model, that is, which training data are used to train the updated target model.

[0093] S306: Update the target model updated by the first update information based on the first update information and the second update information to obtain target update information corresponding to the target model.

[0094] Specifically, after receiving the second update information broadcast by other nodes, the target node will first compare the first update information corresponding to its own training update with the second update information corresponding to the training update of other nodes, and then select the latest update information to replace the first update information as the target update information corresponding to the target model in the target node. For example, but not limited to, comparing the version information of the target model in each update information, and then determining the latest version information (i.e., the one involving the most training data), and updating the target model updated by the first update information to the target model updated by the latest version information, thereby efficiently realizing model training through the update information of the target model shared by multiple nodes in the peer-to-peer computing network. There is no need for each node to train all the training data in the training data set, nor is there a need for each node to have strong computing power. It can even use the computing power of the devices of the majority of netizens (such as mobile phones, computers, etc.) to realize large-scale and efficient model training, thereby improving the performance of model training.

[0095] Optionally, the target node can automatically trigger the synchronization of update information of each node. For example, but not limited to, when multiple nodes in a peer-to-peer computing network jointly complete the training of 50 or 100 training data, it can, but is not limited to, broadcast the first update information obtained after updating its own target model in the peer-to-peer computing network, and trigger other nodes to broadcast the second update information obtained after the corresponding updated target model, etc.

[0096] In some possible embodiments, when the computing power required for training a specified model that the user wants to train is greater than the target threshold, it can be considered that a single device (server or node) is insufficient to complete the training of the specified model or the required training cost is high. In this case, Figure 4 The model training system shown can reduce the training cost and increase the training speed. Figure 4As shown, the model training system may include a server 410 and multiple peer computing networks, each of which may include multiple nodes 110. The server 410 may first divide the specified model into multiple sub-models, and distribute the multiple sub-models as independent target models to multiple peer computing networks for training. That is, before broadcasting the training data set corresponding to the target model, the target node 110 in each peer computing network will first receive the target model and the training data set corresponding to the target model sent by the server 410, and then broadcast the above target model and the corresponding training data set in the peer computing network. In addition, after the target node 110 in each peer computing network updates the target model corresponding to the first update information based on the first update information and the second update information, and obtains the target update information corresponding to the target model, it will also send the target update information corresponding to the target model to the server 410, so that the server 410 aggregates the target update information sent by the target nodes in multiple peer computing networks to obtain the designated update information corresponding to the target designated model, thereby realizing the training of large models efficiently and at low cost.

[0097] The network may be a medium that provides a communication link between the server 410 and any node 110 in any peer computing network, or may be the Internet including network devices and transmission media, but is not limited thereto. The transmission medium may be a wired link, such as but not limited to coaxial cable, optical fiber, and digital subscriber line (DSL), or a wireless link, such as but not limited to wireless fidelity (WIFI), Bluetooth, and mobile device networks.

[0098] It can be understood that the target models and / or training data sets issued by the corresponding servers of multiple peer computing networks may be the same or different, and the embodiments of the present application do not limit this.

[0099] Understandably, Figure 4 The number of peer computing networks in the model training system shown and the number of nodes 110 in each peer computing network are only examples. The number of nodes 110 in each peer computing network may be the same or different, and the embodiments of this specification do not specifically limit this.

[0100] Please refer to the following Figure 5 , which is a schematic diagram of the structure of a model training device provided by an exemplary embodiment of the present application. The above-mentioned model training device is applied to a target node in a peer-to-peer computing network, such as Figure 5 As shown, the model training device 500 includes:

[0101] The first broadcast module 510 is used to broadcast the training data set corresponding to the target model in the above-mentioned peer computing network, so that multiple nodes in the above-mentioned peer computing network negotiate and allocate the above-mentioned training data set, and determine that each of the above-mentioned multiple nodes corresponds to a training data subset to be trained; the above-mentioned training data set includes multiple training data, and the above-mentioned training data subset includes at least one of the above-mentioned training data;

[0102] A training module 520 is used to train the target model based on the target training data subset allocated to the target node to obtain a first training result corresponding to the target model;

[0103] A second broadcast module 530, configured to broadcast the first training result in the peer computing network;

[0104] A first receiving module 540 is used to receive a second training result broadcasted by other nodes in the above-mentioned peer computing network; the above-mentioned second training result is obtained by other nodes in the above-mentioned peer computing network training the above-mentioned target model based on the corresponding allocated training data subset;

[0105] The first updating module 550 is used to update the target model based on the first training result and / or the second training result to obtain first updating information corresponding to the target model.

[0106] In a possible implementation, the model training device 500 further includes:

[0107] A third broadcast module, used to broadcast the first update information in the peer-to-peer computing network;

[0108] A second receiving module is used to receive second update information corresponding to the target model broadcasted by other nodes in the above-mentioned peer-to-peer computing network; the above-mentioned second update information is obtained by other nodes in the above-mentioned peer-to-peer computing network updating the above-mentioned target model based on the above-mentioned first training result broadcasted by the above-mentioned target node and / or the above-mentioned second training result obtained by corresponding training of other nodes in the above-mentioned peer-to-peer computing network;

[0109] The second updating module is used to update the target model updated corresponding to the first updating information based on the first updating information and the second updating information, and obtain the target updating information corresponding to the target model.

[0110] In a possible implementation, the target model is a sub-model of a specified model, and the sub-model is obtained by splitting the specified model by the server;

[0111] The above-mentioned model training device 500 further includes:

[0112] A third receiving module is used to receive the target model and the training data set corresponding to the target model sent by the server;

[0113] A fourth broadcast module, used to broadcast the target model in the peer-to-peer computing network;

[0114] The sending module is used to send the target update information corresponding to the above-mentioned target model to the above-mentioned server, so that the above-mentioned server aggregates the above-mentioned target update information sent by the target nodes in multiple above-mentioned peer computing networks to obtain the specified update information corresponding to the above-mentioned target specified model.

[0115] In a possible implementation, the first updating module 550 includes:

[0116] A determination unit, used to determine whether the current computing power of the target node is greater than the first computing power value;

[0117] A first updating unit, configured to update the target model based on the first training result and the second training result to obtain first update information corresponding to the target model if the current computing power of the target node is greater than the first computing power value;

[0118] A second updating unit is configured to update the target model based on the first training result if the current computing power of the target node is less than or equal to the first computing power value, and obtain first update information corresponding to the target model when the current computing power is greater than the second computing power value;

[0119] Among them, the above-mentioned second computing power value is less than the above-mentioned first computing power value.

[0120] In a possible implementation, the model training device 500 further includes:

[0121] A signature verification module, configured to verify the signature carried by the second training result based on the signature certificates corresponding to the other nodes that broadcast the second training result, and obtain a signature verification result corresponding to the second training result;

[0122] The first update module 550 is specifically used for: if the signature verification result is passed, executing the step of updating the target model based on the first training result and / or the second training result to obtain the first update information corresponding to the target model.

[0123] In one possible implementation, the above-mentioned peer computing network includes the above-mentioned multiple nodes; the above-mentioned multiple nodes have passed the signature verification based on the corresponding signature certificates; the communication protocol versions corresponding to the above-mentioned multiple nodes are the same; and the current computing powers corresponding to the above-mentioned multiple nodes are all greater than the third computing power value.

[0124] In a possible implementation, the training module 520 includes:

[0125] A parameter dimension adjustment unit, used for adjusting the parameter dimension of the target model to be trained currently based on the length of each training data in the target training data subset allocated to the target node;

[0126] The training unit is used to train the above-mentioned target model after adjusting the parameter dimension based on the target training data subset allocated to the above-mentioned target node, so as to obtain a first training result corresponding to the above-mentioned target model.

[0127] The division of each module in the above-mentioned model training device is only for illustration. In other embodiments, the model training device can be divided into different modules as needed to complete all or part of the functions of the above-mentioned model training device. The implementation of each module in the model training device provided in the embodiment of this specification can be in the form of a computer program. The computer program can be run on a terminal or a server. The program modules constituted by the computer program can be stored in the memory of the terminal or the server. When the computer program is executed by the processor, all or part of the steps of the model training method described in the embodiment of this specification are implemented.

[0128] See next Figure 6 , which is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of this specification. Figure 6 As shown, the electronic device 600 may include: at least one processor 610 , at least one communication bus 620 , a user interface 630 , at least one network interface 640 , and a memory 650 .

[0129] The communication bus 620 may be used to realize the connection and communication among the above-mentioned components.

[0130] The user interface 630 may include a display screen (Display) and a camera (Camera), and the optional user interface 630 may also include a standard wired interface and a wireless interface.

[0131] The network interface 640 may optionally include a Bluetooth module, a Near Field Communication (NFC) module, a Wireless Fidelity (Wi-Fi) module, and the like.

[0132] Among them, the processor 610 may include one or more processing cores. The processor 610 uses various interfaces and lines to connect various parts within the entire electronic device 600, and executes various functions and processes data of the routing electronic device 600 by running or executing instructions, programs, code sets or instruction sets stored in the memory 650, and calling data stored in the memory 650. Optionally, the processor 610 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 610 can integrate one or a combination of a processor (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 610, and it can be implemented separately through a chip.

[0133] The memory 650 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 650 includes a non-transitory computer-readable medium. The memory 650 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 650 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a broadcast function, a training function, an update function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments, etc. The memory 650 may optionally be at least one storage device located away from the aforementioned processor 610. As Figure 6 As shown, the memory 650 as a computer storage medium may include an operating system, a network communication module, a user interface module, and program instructions.

[0134] In some possible embodiments, the electronic device 600 may be a target node in a peer-to-peer computing network, or may be the aforementioned Figure 5In the model training device shown, the processor 610 can be used to call the program instructions stored in the memory 650, and specifically perform the following operations: broadcast a training data set corresponding to the target model in the above-mentioned peer computing network, so that multiple nodes in the above-mentioned peer computing network negotiate and allocate the above-mentioned training data set, and determine that the above-mentioned multiple nodes each correspond to a training data subset to be trained; the above-mentioned training data set includes multiple training data, and the above-mentioned training data subset includes at least one of the above-mentioned training data; based on the target training data subset corresponding to the above-mentioned target node, the above-mentioned target model is trained to obtain a first training result corresponding to the above-mentioned target model; the above-mentioned first training result is broadcast in the above-mentioned peer computing network, and the second training result broadcasted by other nodes in the above-mentioned peer computing network is received; the above-mentioned second training result is obtained by training the above-mentioned target model by other nodes in the above-mentioned peer computing network based on the corresponding allocated training data subset; based on the above-mentioned first training result and / or the above-mentioned second training result, the above-mentioned target model is updated to obtain the first update information corresponding to the above-mentioned target model.

[0135] In some possible embodiments, after the processor 610 executes the updating of the target model based on the first training result and / or the second training result to obtain the first update information corresponding to the target model, it is also used to execute: broadcasting the first update information in the peer computing network, and receiving the second update information corresponding to the target model broadcasted by other nodes in the peer computing network; the second update information is obtained by other nodes in the peer computing network updating the target model based on the first training result broadcasted by the target node and / or the second training result obtained by the corresponding training of other nodes in the peer computing network; updating the target model updated by the first update information based on the first update information and the second update information to obtain the target update information corresponding to the target model.

[0136] In some possible embodiments, the target model is a sub-model of a specified model, and the sub-model is obtained by splitting the specified model by the server;

[0137] Before executing the broadcasting of the training data set corresponding to the target model in the peer-to-peer computing network, the processor 610 is further configured to: receive the target model and the training data set corresponding to the target model sent by the server; broadcast the target model in the peer-to-peer computing network;

[0138] The processor 610 executes the above-mentioned updating of the target model corresponding to the first update information based on the first update information and the second update information, and after obtaining the target update information corresponding to the target model, is also used to execute: sending the target update information corresponding to the target model to the server, so that the server aggregates the target update information sent by the target nodes in the multiple peer computing networks to obtain the designated update information corresponding to the target designated model.

[0139] In some possible embodiments, when the processor 610 executes updating of the target model based on the first training result and / or the second training result to obtain the first update information corresponding to the target model, it is specifically used to execute: determining whether the current computing power of the target node is greater than the first computing power value; if so, updating the target model based on the first training result and the second training result to obtain the first update information corresponding to the target model; if not, updating the target model based on the first training result when the current computing power is greater than the second computing power value to obtain the first update information corresponding to the target model; wherein the second computing power value is less than the first computing power value.

[0140] In some possible embodiments, after the processor 610 receives the second training result broadcasted by other nodes in the peer computing network, before executing the step of updating the target model based on the first training result and / or the second training result to obtain the first update information corresponding to the target model, the processor 610 is also used to execute: based on the signature certificate corresponding to the other nodes that broadcast the second training result, verify the signature carried by the second training result to obtain the signature verification result corresponding to the second training result; if the signature verification result is verified, execute the step of updating the target model based on the first training result and / or the second training result to obtain the first update information corresponding to the target model.

[0141] In some possible embodiments, the above-mentioned peer computing network includes the above-mentioned multiple nodes; the above-mentioned multiple nodes have passed the signature verification based on the corresponding signature certificates; the communication protocol versions corresponding to the above-mentioned multiple nodes are the same; and the current computing powers corresponding to the above-mentioned multiple nodes are all greater than the third computing power value.

[0142] In some possible embodiments, when the processor 610 executes training of the target model based on the target training data subset assigned to the target node to obtain the first training result corresponding to the target model, it is specifically used to execute: adjusting the parameter dimension of the target model currently to be trained based on the length of each training data in the target training data subset assigned to the target node; training the target model after the parameter dimension is adjusted based on the target training data subset assigned to the target node to obtain the first training result corresponding to the target model.

[0143] The embodiment of the present application also provides a computer storage medium, which stores instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes one or more steps in any of the above methods. If the components of the above model training device are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above storage medium.

[0144] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The above computer program product includes one or more computer instructions. When the above computer program instructions are loaded and executed on a computer, the above process or function according to the embodiment of the present application is generated in whole or in part. The above computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above computer instructions can be stored in a computer-readable storage medium or transmitted by the above computer-readable storage medium. The above computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The above computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The above available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

[0145] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes. In the absence of conflict, the technical features in this embodiment and the implementation scheme can be combined arbitrarily.

[0146] The above-mentioned embodiments are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements made to the technical solutions of the present application by ordinary technicians in this field should fall within the protection scope determined by the claims of the present application.

Claims

1. A model training method, characterized in that: The method is applied to a target node in a peer-to-peer computing network, and the method comprises: Broadcasting a training data set corresponding to the target model in the peer-to-peer computing network, so that multiple nodes in the peer-to-peer computing network negotiate and allocate the training data set, and determining that each of the multiple nodes corresponds to a training data subset to be trained; the training data set includes multiple training data, and the training data subset includes at least one of the training data; The target model is trained based on the target training data subset corresponding to the target node to obtain a first training result corresponding to the target model; the number of training data in the target training data subset corresponding to the target node is updated based on the update of the number of nodes in the peer computing network; Broadcasting the first training result in the peer-to-peer computing network, and receiving a second training result broadcasted by other nodes in the peer-to-peer computing network; the second training result is obtained by training the target model by other nodes in the peer-to-peer computing network based on the corresponding allocated training data subset; Update the target model based on the first training result and / or the second training result to obtain first update information corresponding to the target model; The updating of the target model based on the first training result and / or the second training result to obtain first update information corresponding to the target model includes: If the current computing power of the target node is greater than the first computing power value, updating the target model based on the first training result and the second training result to obtain first update information corresponding to the target model; If the current computing power of the target node is less than or equal to the first computing power value, then when the current computing power is greater than the second computing power value, updating the target model based on the first training result to obtain first update information corresponding to the target model; If the current computing power of the target node is less than or equal to the second computing power value, receiving second update information broadcasted by other nodes in the peer computing power network after updating the target model based on the first training result and / or the second training result, and updating the target model to the target model in the second update information; Among them, the second computing power value is less than the first computing power value; the first computing power value is any value greater than or equal to the total computing power required to update the target model based on the first training result and the second training result; the second computing power value is any value greater than or equal to the total computing power required to update the target model based on the first training result and less than the first computing power value.

2. The method according to claim 1, characterized in that After updating the target model based on the first training result and / or the second training result to obtain first update information corresponding to the target model, the method further includes: Broadcasting the first update information in the peer-to-peer computing network, and receiving second update information corresponding to the target model broadcasted by other nodes in the peer-to-peer computing network; the second update information is obtained by other nodes in the peer-to-peer computing network updating the target model based on the first training result broadcasted by the target node and / or the second training result obtained by corresponding training of other nodes in the peer-to-peer computing network; The target model updated by the first update information is updated based on the first update information and the second update information to obtain target update information corresponding to the target model.

3. The method according to claim 2, characterized in that The target model is a sub-model of the specified model, and the sub-model is obtained by splitting the specified model by the server; Before broadcasting the training data set corresponding to the target model in the peer computing network, the method further includes: Receiving the target model and a training data set corresponding to the target model sent by the server; Broadcasting the target model in the peer-to-peer computing network; After updating the target model updated by the first update information based on the first update information and the second update information, and obtaining the target update information corresponding to the target model, the method further includes: Send the target update information corresponding to the target model to the server, so that the server aggregates the target update information sent by the target nodes in the multiple peer computing networks to obtain the specified update information corresponding to the specified model.

4. The method according to claim 1, characterized in that: After receiving the second training result broadcasted by other nodes in the peer computing network, and before updating the target model based on the first training result and / or the second training result and obtaining first update information corresponding to the target model, the method further includes: Verifying the signature carried by the second training result based on the signature certificates corresponding to the other nodes that broadcast the second training result, to obtain a signature verification result corresponding to the second training result; If the signature verification result is passed, the step of updating the target model based on the first training result and / or the second training result to obtain first update information corresponding to the target model is performed.

5. The method according to claim 1, characterized in that The peer computing network includes the multiple nodes; the multiple nodes have passed the signature verification based on the corresponding signature certificates; the communication protocol versions corresponding to the multiple nodes are the same; and the current computing powers corresponding to the multiple nodes are all greater than the third computing power value.

6. The method according to claim 1, characterized in that The step of training the target model based on the target training data subset allocated to the target node to obtain a first training result corresponding to the target model includes: Adjusting the parameter dimension of the target model to be trained currently based on the length of each training data in the target training data subset allocated to the target node; The target model after adjusting the parameter dimension is trained based on the target training data subset allocated to the target node to obtain a first training result corresponding to the target model.

7. A model training device, characterized in that: The model training device is applied to a target node in a peer-to-peer computing network, and the model training device includes: A first broadcast module is used to broadcast a training data set corresponding to a target model in the peer-to-peer computing network, so that a plurality of nodes in the peer-to-peer computing network negotiate and allocate the training data set, and determine that each of the plurality of nodes corresponds to a training data subset to be trained; the training data set includes a plurality of training data, and the training data subset includes at least one of the training data; A training module, configured to train the target model based on the target training data subset corresponding to the target node, and obtain a first training result corresponding to the target model; the number of training data in the target training data subset corresponding to the target node is updated based on the update of the number of nodes in the peer computing network; A second broadcast module, configured to broadcast the first training result in the peer computing network; A first receiving module is used to receive a second training result broadcasted by other nodes in the peer-to-peer computing network; the second training result is obtained by training the target model by other nodes in the peer-to-peer computing network based on the corresponding allocated training data subset; A first updating module, configured to update the target model based on the first training result and / or the second training result, and obtain first updating information corresponding to the target model; The first update module is specifically used for: if the current computing power of the target node is greater than the first computing power value, updating the target model based on the first training result and the second training result to obtain first update information corresponding to the target model; if the current computing power of the target node is less than or equal to the first computing power value, then when the current computing power is greater than the second computing power value, updating the target model based on the first training result to obtain first update information corresponding to the target model; if the current computing power of the target node is less than or equal to the second computing power value, receiving the second update information broadcasted by other nodes in the peer computing power network after updating the target model based on the first training result and / or the second training result, and updating the target model to the target model in the second update information; Among them, the second computing power value is less than the first computing power value; the first computing power value is any value greater than or equal to the total computing power required to update the target model based on the first training result and the second training result; the second computing power value is any value greater than or equal to the total computing power required to update the target model based on the first training result and less than the first computing power value.

8. The device according to claim 7, characterized in that The model training device also includes: A third broadcast module, configured to broadcast the first update information in the peer computing network; A second receiving module is used to receive second update information corresponding to the target model broadcasted by other nodes in the peer-to-peer computing network; the second update information is obtained by updating the target model by other nodes in the peer-to-peer computing network based on the first training result broadcasted by the target node and / or the second training result obtained by corresponding training of other nodes in the peer-to-peer computing network; The second updating module is used to update the target model updated corresponding to the first updating information based on the first updating information and the second updating information, and obtain the target updating information corresponding to the target model.

9. The device according to claim 8, characterized in that The target model is a sub-model of the specified model, and the sub-model is obtained by splitting the specified model by the server; The model training device also includes: A third receiving module is used to receive the target model and the training data set corresponding to the target model sent by the server; A fourth broadcast module, configured to broadcast the target model in the peer computing network; A sending module is used to send the target update information corresponding to the target model to the server, so that the server aggregates the target update information sent by the target nodes in multiple peer computing networks to obtain the specified update information corresponding to the specified model.

10. The device according to claim 7, characterized in that The model training device also includes: a signature verification module, configured to verify the signature carried by the second training result based on the signature certificates corresponding to the other nodes that broadcast the second training result, and obtain a signature verification result corresponding to the second training result; The first update module is specifically used for: if the signature verification result is verified to be passed, executing the step of updating the target model based on the first training result and / or the second training result to obtain first update information corresponding to the target model.

11. The device according to claim 7, characterized in that The peer computing network includes the multiple nodes; the multiple nodes have passed the signature verification based on the corresponding signature certificates; the communication protocol versions corresponding to the multiple nodes are the same; and the current computing powers corresponding to the multiple nodes are all greater than the third computing power value.

12. The device according to claim 7, characterized in that The training module includes: A parameter dimension adjustment unit, configured to adjust the parameter dimension of the target model to be trained currently based on the length of each training data in the target training data subset allocated to the target node; A training unit is used to train the target model after adjusting the parameter dimension based on the target training data subset allocated to the target node, so as to obtain a first training result corresponding to the target model.

13. An electronic device, characterized in that: include: processor and memory; wherein, The processor is connected to the memory; the memory is used to store executable program code; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the method steps according to any one of claims 1 to 6.

14. A computer storage medium, characterized in that: The computer storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 6.

Images