Distributed training method, system, device, storage medium and program product

Abstract

The present application discloses a distributed training method, system, device, storage medium and program product, wherein the method comprises: a scheduling platform receives configuration information submitted by a user, generates target scheduling information of each training device according to the initial scheduling information of at least two training devices participating in joint training included in the configuration information; sends the target scheduling information of each training device to the corresponding training device respectively, so that each training device schedules a plurality of algorithm components maintained in advance according to the corresponding target scheduling information, and performs joint training using the scheduled target algorithm components. The present application realizes the flexible scheduling of a plurality of algorithms in distributed training.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a distributed training method, system, device, storage medium, and program product. Background Technology

[0002] Distributed training involves jointly training a machine learning model across multiple training nodes. Federated learning is a typical example of distributed training, which involves various algorithms, such as training algorithms and security algorithms. Currently, in federated training, training is often based on fixed training and security algorithms according to training requirements, making it impossible to flexibly schedule multiple training and security algorithms. Summary of the Invention

[0003] This application addresses the problem in the aforementioned related technologies that cannot flexibly schedule multiple algorithms by providing a distributed training method, system, device, storage medium, and program product.

[0004] The first aspect of this application provides a distributed training method applied to a scheduling platform, the method comprising:

[0005] Receive configuration information submitted by the user; the configuration information includes the initial scheduling information of each of the at least two training devices participating in the joint training, and the initial scheduling information includes partial scheduling information of various algorithm components;

[0006] Based on the initial scheduling information of the training equipment, the target scheduling information of the training equipment is generated; the target scheduling information includes complete scheduling information for various algorithm components.

[0007] The target scheduling information of the training device is sent to the training device. The target scheduling information is used to schedule multiple algorithm components based on the target scheduling information, and to perform joint training based on the scheduled target algorithm components. The training device pre-maintains multiple algorithm components.

[0008] A second aspect of this application provides a distributed training method applied to a training device, the method comprising:

[0009] The system receives target scheduling information sent by the scheduling platform. The target scheduling information is generated by the scheduling platform after receiving configuration information submitted by the user, based on the initial scheduling information of each of the at least two training devices participating in the joint training, which is included in the configuration information. The initial scheduling information includes partial scheduling information of various algorithm components, and the target scheduling information includes complete scheduling information of various algorithm components.

[0010] Based on the target scheduling information, various pre-maintained algorithm components are scheduled to obtain the target algorithm component;

[0011] Joint training is performed using the target algorithm components.

[0012] A third aspect of this application provides a distributed training apparatus for use in a scheduling platform, the apparatus comprising:

[0013] The receiving module is used to receive configuration information submitted by the user. The configuration information includes the initial scheduling information of each of the at least two training devices participating in the joint training. The initial scheduling information includes partial scheduling information of various algorithm components.

[0014] The generation module is used to generate target scheduling information for the training equipment based on the initial scheduling information of the training equipment; the target scheduling information includes complete scheduling information for various algorithm components.

[0015] The sending module is used to send the target scheduling information of the training device to the training device. The target scheduling information is used to schedule multiple algorithm components based on the target scheduling information, and to perform joint training based on the scheduled target algorithm components. The training device pre-maintains multiple algorithm components.

[0016] A fourth aspect of this application provides a distributed training apparatus for use in a training device, the apparatus comprising:

[0017] The receiving module is used to receive target scheduling information sent by the scheduling platform. The target scheduling information is generated by the scheduling platform after receiving the configuration information submitted by the user, based on the initial scheduling information of at least two training devices participating in joint training, which is included in the configuration information. The initial scheduling information includes partial scheduling information of various algorithm components, and the target scheduling information includes complete scheduling information of various algorithm components.

[0018] The scheduling module is used to schedule multiple pre-maintained algorithm components according to the target scheduling information to obtain the target algorithm component;

[0019] The training module is used for joint training using the target algorithm components.

[0020] The fifth aspect of this application provides a distributed training system, which includes: a scheduling platform and multiple training devices;

[0021] The scheduling platform is used to receive configuration information submitted by users. The configuration information includes the initial scheduling information of at least two training devices participating in joint training among multiple training devices; generate target scheduling information for the training devices based on the initial scheduling information of the training devices; and send the target scheduling information of the training devices to the training devices respectively. The initial scheduling information includes partial scheduling information of various algorithm components, and the target scheduling information includes complete scheduling information of various algorithm components.

[0022] The training device is used to receive target scheduling information sent by the scheduling platform; to schedule multiple pre-maintained algorithm components according to the target scheduling information to obtain the target algorithm component; and to perform joint training using the target algorithm component.

[0023] A sixth aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method as described in the first aspect above, or to implement the method as described in the second aspect above.

[0024] A seventh aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method as described in the first aspect above, or to implement the method as described in the second aspect above.

[0025] The eighth aspect of this application provides a computer program product including a computer program that is executed by a processor to implement the method described in the first aspect above, or to implement the method described in the second aspect above.

[0026] Based on the distributed training method provided in the first aspect above, this application has at least the following beneficial effects or advantages:

[0027] In this embodiment, when the scheduling platform receives configuration information input by the user, it generates target scheduling information for each training device based on the initial scheduling information of at least two training devices participating in joint training, included in the configuration information. The target scheduling information for each training device is then sent to the corresponding training device, enabling each training device to schedule pre-maintained multiple algorithm components according to the corresponding target scheduling information and perform joint training using the scheduled target algorithm components. Thus, by pre-constructing multiple algorithm components and having each training device maintain them, and by providing configuration functionality through the scheduling platform, the platform can determine target scheduling information based on the initial scheduling information input by the user, and then send the target scheduling information to the training devices to schedule the multiple algorithm components maintained by the training devices. This training method achieves flexible scheduling of multiple algorithms. Furthermore, since the training devices can directly perform joint training based on the scheduled target algorithm components without needing to write code based on corresponding training requirements, training efficiency is greatly improved.

[0028] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0029] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0030] Figure 1 This is a schematic diagram illustrating an application scenario of a distributed training method provided in an embodiment of this application;

[0031] Figure 2 This is a first flowchart of a distributed training method provided in an embodiment of this application;

[0032] Figure 3 A second flowchart of a distributed training method provided in an embodiment of this application;

[0033] Figure 4 A third flowchart of a distributed training method provided in an embodiment of this application;

[0034] Figure 5 A fourth flowchart of a distributed training method provided in this application embodiment;

[0035] Figure 6 A fifth flowchart of a distributed training method provided in an embodiment of this application;

[0036] Figure 7 A schematic diagram of the training process for the interactive training mode provided in the embodiments of this application;

[0037] Figure 8 A schematic diagram of the training process for the centralized training mode provided in the embodiments of this application;

[0038] Figure 9 This is a schematic diagram of a first structure of a distributed training device provided in an embodiment of this application;

[0039] Figure 10 This is a second structural schematic diagram of a distributed training device provided in an embodiment of this application;

[0040] Figure 11 This is a schematic diagram of the composition of a distributed training system provided in an embodiment of this application;

[0041] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0042] Figure 13 This is a schematic diagram of the structure of a storage medium provided in an embodiment of this application. Detailed Implementation

[0043] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0044] First, some of the nouns or terms that appear in the description of the embodiments in this application shall be interpreted as follows:

[0045] Federated Learning (FL) is a machine learning technique that allows multiple training devices to jointly train a model without sharing data. It can improve model accuracy and training efficiency while protecting data privacy.

[0046] Differential Privacy (DP): A data privacy protection technology that protects data privacy by adding noise to the data.

[0047] Homomorphic encryption (HE) is an encryption technique that can encrypt and decrypt data while preserving its structure and original meaning. It can improve data security while protecting data privacy.

[0048] Multi-party secure computing (MPC) is a data privacy protection technology that can break down a piece of data into multiple parts. Each part cannot be restored to the original data, but after separate computation, the result of centralized computation can be achieved. The decrypted result can be used for federated learning.

[0049] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0050] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination," etc.

[0051] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0052] Federated learning, as a distributed machine learning method that protects data privacy, enables collaborative training of models without the raw data leaving the local device. This means that federated learning utilizes multiple algorithms, including security algorithms and training algorithms. While federated learning provides stronger protection for data privacy and security, current methods do not support the flexible scheduling of multiple security and training algorithms. Specifically, when developers receive training requirements, they often write code based on the specified security and training algorithms. These security and training algorithms are coupled and cannot be scheduled. However, if the training requirements change, code adjustments are necessary, which often takes considerable time, reducing training efficiency. Furthermore, different training requirements may share the same security or training algorithms. Writing code for each training requirement not only results in low code reusability but also increases the amount of code that needs to be maintained.

[0053] Based on this, embodiments of this application provide a distributed training method. Figure 1 This is a schematic diagram illustrating an application scenario of a distributed training method provided in an embodiment of this application. For example... Figure 1As shown, this scenario includes a scheduling platform and multiple training devices. The scheduling platform can be located on a user's terminal device or on a server, and it communicates with each training device. Training devices can be either terminal devices or servers. Each training device maintains multiple algorithm components, which may or may not be located on the training device itself, such as in the cloud. Terminal devices can be mobile phones, tablets, desktop computers, laptops, etc. Servers can be physical servers, cloud servers, etc. When the scheduling platform is located on a terminal device, it can be a standalone application (App), a mini-program within an application, or a web application. When the scheduling platform is located on a server, the user's terminal device can have a client configured to communicate with the scheduling platform. This client can be a standalone application, a mini-program within an application, or a web application. This client is used to send the user-input configuration information to the scheduling platform.

[0054] Figure 1 The example shown uses two devices: a scheduling platform located on the user's desktop computer and training equipment participating in joint training. It should be understood that... Figure 1 The illustration is merely a representative representation of an application scenario for the distributed training method involved in this application and does not constitute a limitation on the technical solution of this application. In other embodiments, the application scenario of the distributed training method involved in this application may include more or fewer components.

[0055] Specifically, in Figure 1The scheduling platform can include a human-computer interaction interface, including a login interface and a configuration interface. The login interface is used for users to log in to the scheduling platform, and the configuration interface is used for users to edit the configuration information for distributed training. The scheduling platform can maintain login information for at least one user, information on algorithms the user has access to, device information for training devices the user has access to, sample information for training samples corresponding to each training device, a target scheduling information generation strategy, and a target scheduling information transmission strategy. Device information includes device identifiers and communication addresses. Sample information includes label information and database information for the training samples. The target scheduling information generation strategy instructs the scheduling platform to generate target scheduling information for each training device based on the initial scheduling information of at least two training devices participating in joint training, as included in the configuration information input by the user. The target scheduling information transmission strategy instructs the scheduling platform to send the generated target scheduling information to the corresponding training devices participating in joint training. Each training device maintains pre-built multiple algorithm components, their scheduling strategies, and training strategies. The system comprises multiple algorithm components, including multiple security components in a security component set, multiple training components in a training component set, and multiple verification components in a verification component set. Each security component executes a security algorithm, each training component executes a training algorithm, and each verification component executes a verification algorithm. A scheduling strategy instructs the training device to schedule the pre-maintained multiple algorithm components according to received target scheduling information. A training strategy instructs the training device to perform joint training based on the scheduled target algorithm components.

[0056] As can be seen, by pre-constructing multiple algorithm components and having each training device maintain these components, and by providing configuration functionality through the scheduling platform, the platform can determine target scheduling information based on the initial scheduling information input by the user. This target scheduling information is then sent to the training devices to schedule the various algorithm components maintained by the training devices. Therefore, this training method achieves flexible scheduling of multiple algorithms. Furthermore, since the training devices can directly perform joint training based on the scheduled target algorithm components without needing to write code based on specific training requirements, training efficiency is greatly improved.

[0057] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0058] Figure 2 A flowchart of the distributed training method provided in the embodiments of this application. Figure 2The method shown can be used by Figure 1 The scheduling platform in the process executes the commands. For example... Figure 2 As shown, the method includes the following steps 101-103.

[0059] Step 101: Receive configuration information submitted by the user; the configuration information includes the initial scheduling information of at least two training devices participating in joint training, and the initial scheduling information includes partial scheduling information of various algorithm components.

[0060] Step 102: Generate target scheduling information for each training device based on the initial scheduling information of each training device. The target scheduling information includes complete scheduling information for various algorithm components.

[0061] Step 103: Send the target scheduling information of each training device to the corresponding training device. The target scheduling information is used to schedule multiple algorithm components based on the target scheduling information, and to perform joint training using the scheduled target algorithm components. The training device pre-maintains multiple algorithm components.

[0062] In some implementations, the scheduling platform can be located on the user's terminal device. Accordingly, when a user has distributed training requirements, they can operate the scheduling platform on their terminal device to input configuration information and submit it. In response to the user's submission, the scheduling platform obtains the configuration information submitted by the user; and based on the initial scheduling information of each of the at least two training devices participating in the joint training, which is included in the configuration information, it generates target scheduling information for each training device and sends the target scheduling information to the corresponding training device.

[0063] In other implementations, the scheduling platform can be located on the server side, and the user's terminal device can be configured as a client that communicates with the scheduling platform. Accordingly, when a user has distributed training requirements, they can input configuration information and submit it via the client on their terminal device. The client responds to the user's submission by obtaining the submitted configuration information and sending it to the scheduling platform. Based on the received configuration information, including the initial scheduling information of at least two training devices participating in the joint training, the scheduling platform generates target scheduling information for each training device and sends the target scheduling information to the corresponding training device.

[0064] When the training device receives target scheduling information from the scheduling platform, it schedules multiple pre-maintained algorithm components according to the target scheduling information to obtain the target algorithm component; then, it uses the target algorithm component for joint training. Each of the multiple algorithm components is pre-written with code based on the algorithm it will execute, and this code is then encapsulated. Understandably, since the same algorithm component can be used in different training scenarios without needing to write corresponding code for each scenario, code reusability is improved, and the amount of code that needs to be maintained is reduced.

[0065] The initial scheduling information mentioned above may include the device identifiers, security algorithms, training algorithms, model validation algorithms, and sample information of each of the at least two training devices participating in joint training. The target scheduling information may add information such as training mode and device communication addresses to the initial scheduling information. That is, the initial scheduling information includes partial scheduling information for various algorithm components, while the target scheduling information includes complete scheduling information for various algorithm components. The security algorithms may include a first security algorithm and a second security algorithm. The first security algorithm refers to the algorithm used when the training devices participating in joint training interact with each other, and the second security algorithm refers to the algorithm used to perform privacy set intersection on the training samples. The training algorithm is the algorithm used in model training. The model validation algorithm is the algorithm used to validate the model's performance. The first security algorithm may be any one of the following: secp256k1 (Secure Elliptic Curve 256k1), Curve25519, or Ed25519 (Edwards-curve Digital Signature Algorithm (EdDSA) over Curve25519). The second security algorithm can be any one of homomorphic encryption, differential privacy, or multi-party secure computation. The training algorithm can include any one of logistic regression, linear regression, decision trees, or random forest. The model validation algorithm can include any one of cross-validation, recall, or mean squared error (MSE).

[0066] In distributed training, at least two devices participating in joint training can be categorized as master devices (also known as host devices) and slave devices (also known as slave devices). The master device holds the first labeled training samples, while the slave device holds the second unlabeled training samples. During joint training, the master and slave devices perform different operations. Therefore, the scheduling platform generates target configuration information for each participating training device based on the initial scheduling information of each device included in the user-input configuration information. The operations performed by the master and slave devices during training are described in detail below.

[0067] In the distributed training method provided in this application embodiment, when the scheduling platform receives configuration information submitted by the user, it generates target scheduling information for each training device based on the initial scheduling information of at least two training devices participating in joint training, which is included in the configuration information. The target scheduling information for each training device is then sent to the corresponding training device, enabling each training device to schedule multiple pre-maintained algorithm components according to the corresponding target scheduling information and perform joint training using the scheduled target algorithm components. Thus, by pre-constructing multiple algorithm components and having each training device maintain these components, and by providing configuration functionality through the scheduling platform, the platform can determine the target scheduling information based on the initial scheduling information input by the user, and then send the target scheduling information to the training devices to schedule the multiple algorithm components maintained by the training devices. It is evident that this training method achieves flexible scheduling of multiple algorithms. Furthermore, since the training devices can directly perform joint training based on the scheduled target algorithm components without needing to write code based on corresponding training requirements, training efficiency is greatly improved.

[0068] To facilitate the management of training devices and algorithms accessible to each user, in some implementations, each user can first register on the scheduling platform and, after successful registration, log in to their registered account to add or delete their own devices and algorithms. Specifically, the method may also include the following steps A1 and A2:

[0069] Step A1: Obtain the user's configuration item processing request; the configuration item processing request is used to request the addition and / or deletion of the target configuration item; the target configuration item includes at least one of training device, training sample, security algorithm, training algorithm, and model verification algorithm.

[0070] Step A2: Based on the configuration item processing request, process the target configuration item accordingly.

[0071] In some implementations, the configuration item processing request is used to request the addition of a training device. Accordingly, after successfully logging into the scheduling platform, the user can edit the device information of at least one training device to be added, the sample information of the corresponding training samples, etc., in the configuration item addition interface of the scheduling platform, and submit it to send a configuration item processing request to the scheduling platform. Correspondingly, the scheduling platform responds to the user's submission operation, obtains the configuration item processing request submitted based on the addition interface, and establishes a first association between the device information and sample information included in the configuration item processing request and the user's user information (such as successfully registered account information, user identifier, etc.), and saves this first association to indicate that the user has access rights to the device corresponding to the device information. The device information may include a device identifier, communication address, etc. The communication address is used to locate the training device and conduct data communication with the training device; the communication address can be an IP (Internet Protocol) address, a MAC (Media Access Control) address, etc. The sample information may include the database information of the database where the training samples are located, such as the database access address, the database identifier, etc. When the training samples corresponding to the training device carry tags, the sample information may also include tag information. It should be noted that each training device can correspond to at least one sample information, meaning that each training device can access at least one training sample in the database. Training devices that a user has access to include training devices added by the user's operation scheduling platform, training devices added by other user operation scheduling platforms and granted access to that user, etc.

[0072] Considering that in practical applications, users may no longer use at least one training device they have access to for distributed training for various reasons, in this case, users can also operate the scheduling platform to delete the corresponding training device from the scheduling platform. That is, in some implementations, a configuration item processing request is used to request the deletion of a training device. Accordingly, after successfully logging into the scheduling platform, the user can edit (e.g., select from multiple training devices displayed on the deletion interface) the device information of at least one training device to be deleted in the scheduling platform's configuration item deletion interface and submit it to send a configuration item processing request to the scheduling platform. Correspondingly, the scheduling platform responds to the user's submission operation, obtains the configuration item processing request submitted by the user based on the deletion page, retrieves the corresponding target association from the saved first association based on the device information included in the configuration item processing request, and deletes the target association.

[0073] When the target configuration items are training samples, security algorithms, training algorithms, and model verification algorithms, the process of adding and deleting them is similar to the process of adding and deleting training devices described above. Please refer to the relevant descriptions above, and they will not be listed and explained one by one here.

[0074] It should be noted that when the target configuration item includes multiple items (such as training devices and security algorithms), some of the configuration items (such as training devices) can be added, while others (such as security algorithms) can be deleted. When there is only one target configuration item (such as training devices), either adding or deleting the target configuration item is performed. When the target configuration item is a training sample, if a training sample is added, the configuration item processing request can include the database information of the training sample and the device information of the training device corresponding to the training sample; if a training sample is deleted, the configuration item processing request can include the database information of the training sample. When the target configuration item is a security algorithm, training algorithm, or model validation algorithm, the configuration item processing request can include at least one of the following: algorithm name, algorithm identifier, etc.

[0075] It should be further noted that when the target configuration item is a training device or training sample, the configuration item processing request can also be used to request modification of the target configuration item. For example, modifying the communication address of the training device because its geographical location has changed, or modifying the database information of the training sample because it has been migrated to another database.

[0076] Therefore, the scheduling platform provides users with the ability to add and delete training devices, training samples, and various algorithms. This not only ensures that users can flexibly schedule multiple algorithms and training devices, but also enables users to conveniently manage their training devices and various algorithms. At the same time, it also facilitates the scheduling platform to manage the training devices and algorithms that each user has access to.

[0077] To flexibly and accurately schedule multiple algorithms and devices with user access permissions during distributed training, in some implementations, after successfully logging into the scheduling platform, the user submits configuration information, including initial scheduling information, through the platform's configuration interface. Specifically, for example... Figure 3 As shown, the steps 1001 and 1002 may be included before the aforementioned step 101.

[0078] Step 1001: If the login information submitted by the user to the scheduling platform is valid, then obtain the device information of multiple training devices that the user has access to and the algorithm information of multiple algorithms.

[0079] Specifically, before performing configuration operations, users can first open the scheduling platform. In response to the user's open operation, the scheduling platform displays a login interface and retrieves the login information submitted by the user. The retrieved login information is matched against pre-saved login information; if the match is successful, the login information submitted by the user to the scheduling platform is deemed valid. Furthermore, based on the user's information, multiple associated device information and multiple algorithm information are retrieved from the saved first association relationship.

[0080] Step 1002: Display the configuration interface based on the acquired multiple device information and multiple algorithm information. The configuration interface is used to display multiple device information and multiple algorithm information.

[0081] The multiple device information and multiple algorithm information can be set in the configuration interface in a tiled manner or in a drop-down menu manner. This application does not make specific limitations on the specific form of the configuration interface.

[0082] Corresponding to steps 1001 and 1002, such as Figure 3 As shown, step 101 may include the following step 1011:

[0083] Step 1011: Receive configuration information submitted by the user based on the configuration interface; the configuration information includes the initial scheduling information of at least two training devices participating in the joint training.

[0084] Providing users with a configuration interface improves the convenience of configuration operations and also helps to accurately obtain users' configuration information.

[0085] After obtaining the user configuration information, target scheduling information for each training device participating in the joint training can be generated based on the configuration information. Specifically, such as... Figure 4 As shown, step 102 may include steps 1021 and 1022.

[0086] Step 1021: Based on the initial scheduling information of each training device, determine the common configuration information and private configuration information corresponding to each training device.

[0087] Considering that in distributed training, the security algorithm, model verification algorithm, and training mode are usually the same for the master and slave devices, in order to improve the efficiency and accuracy of determining common configuration information, in some implementations, step 1021, based on the initial scheduling information of each training device, determines the common configuration information corresponding to each training device. This may include: obtaining the security algorithm, training algorithm, and model verification algorithm from the initial scheduling information; determining the training mode based on the security algorithm and training algorithm; and determining the security algorithm, model verification algorithm, and training mode as the common configuration information corresponding to each training device. In other words, the common configuration information is the same configuration information for at least two training devices participating in joint training.

[0088] Determining the training mode based on the security algorithm and the training algorithm may include: obtaining the associated training mode from the second correlation between the security algorithm, the training algorithm, and the training mode, based on the security algorithm and the training algorithm. For example, if the security algorithm is a homomorphic encryption algorithm and the training algorithm is a logistic regression algorithm, then the determined training mode is an interactive training mode.

[0089] Considering that in distributed training, the training samples and communication addresses corresponding to the master device and slave devices are often different, in order to ensure the accuracy of the determined private information, in some implementations, step 1021, based on the initial scheduling information of each training device, determines the private configuration information corresponding to each training device. This may include: determining the training algorithm and the communication address corresponding to each training device based on the initial scheduling information of each training device; determining the sample information of the training samples corresponding to each training device; for any training device, if the training device is the master device in the joint training, then the training algorithm, the sample information corresponding to the training device, and the communication address of the slave device are determined as the private configuration information corresponding to the training device; if the training device is the slave device in the joint training, then the sample information of the training device and the communication address of the master device are determined as the private configuration information corresponding to the training device. In other words, the private configuration information consists of different configuration information for at least two training devices participating in the joint training.

[0090] Specifically, the training algorithm, device identifiers of each training device participating in the joint training, and sample information corresponding to each training device are obtained from the initial scheduling information. For each training device participating in the joint training, the associated communication address is obtained from the saved first association relationship based on the device identifier of the training device, and the obtained communication address is determined as the communication address corresponding to the training device. Furthermore, if it is determined that the sample information corresponding to the training device includes label information, then the training device is determined to be the master device of the joint training; if it is determined that the sample information corresponding to the training device does not include label information, then the training device is determined to be the slave device of the joint training. When the training device is the master device, the training algorithm, the sample information corresponding to the training device, and the communication address of the slave device are determined as the private configuration information corresponding to the training device. When the training device is the slave device, the sample information of the training device and the communication address of the master device are determined as the private configuration information corresponding to the training device.

[0091] For example, the training devices participating in joint training include training device 1 and training device 2, where training device 1 is the master device and training device 2 is the slave device. Then, the training algorithm, the sample information corresponding to training device 1, and the communication address of training device 2 are determined as the private configuration information corresponding to training device 1; and the sample information corresponding to training device 2 and the communication address of training device 1 are determined as the private configuration information corresponding to training device 2.

[0092] Step 1022: Generate target scheduling information for each training device based on the public and private configuration information corresponding to each training device.

[0093] Specifically, for each training device, the public configuration information and private configuration information of the training device are determined as the target scheduling information of the training device.

[0094] In other words, sending the target scheduling information of each training device to the corresponding training device in step 103 may include: sending the common configuration information of the training device and the private configuration information corresponding to the training device to any training device.

[0095] By determining the public and private configuration information corresponding to each training device participating in the joint training, and generating the target configuration information of the training device based on the public and private configuration information corresponding to each training device, the accuracy of the target configuration information is ensured, thereby providing a guarantee for the subsequent accurate scheduling algorithm components.

[0096] Considering that in practical applications, different training samples in the same database may not be identical in terms of structure, form, etc., in order to meet the training requirements. In some embodiments, the user can also send a processing request for the training sample to the corresponding training device through the scheduling platform. Specifically, the method may further include the following steps B1 and step B2:

[0097] Step B1, if the processing information of the target training sample submitted by the user is obtained, determine the target training device corresponding to the target training sample according to the processing information.

[0098] Among them, the processing information may include the access information of the target training sample, and the access information may be the library information of the database where the target training sample is located. The library information may be the database identifier or the access address of the database. Correspondingly, when the scheduling platform obtains the processing information submitted by the user, it obtains the associated device information from the saved first association relationship according to the library information included in the processing information, and determines the training device corresponding to the obtained device information as the target training device corresponding to the target training sample.

[0099] Step B2, send a processing request to the target training device according to the processing information; the processing request is used to request the target training device to process the target training sample according to the target structure standard to obtain a standardized training sample.

[0100] Among them, the target structure standard may be the structure standard specified by the user included in the processing information or a preset structure standard. The target structure standard may include setting the missing data of the first feature of the target training sample to the target data, expanding the second feature of the target training sample from the first form to the second form, etc. For example, the target structure standard is to set the missing data of the "address" feature in the training sample to address 1; another example is that the target structure standard is to expand the "time" feature in the training sample from "yyyy.mm.dd" to "yyyy year mm month dd day". For the target structure standard, no specific limitation is made in this application, and it can be set according to needs in actual applications.

[0101] It should be noted that the training sample can be various types of samples such as images, texts, audio and video, and no specific limitation is made in this application.

[0102] By providing the processing function of the training sample through the scheduling platform, the user only needs to operate the scheduling platform to send a processing request to the corresponding target training device, without the user manually processing the target training sample, which greatly improves the processing efficiency of the standardized training sample.

[0103] Corresponding to the foregoing embodiments of the distributed training method, this application also provides an embodiment of the distributed training method. Figure 5 This is a flowchart illustrating a distributed training method according to an exemplary embodiment. Figure 5 The method shown can be used by Figure 1 The training equipment in the middle is used for execution. For example... Figure 5 As shown, the method includes the following steps 201-203.

[0104] Step 201: Receive target scheduling information sent by the scheduling platform; the target scheduling information is generated by the scheduling platform after receiving the configuration information submitted by the user, based on the initial scheduling information of each of the at least two training devices participating in the joint training, which is included in the configuration information; the initial scheduling information includes partial scheduling information of various algorithm components, and the target scheduling information includes complete scheduling information of various algorithm components.

[0105] Step 202: Based on the target scheduling information, schedule the various pre-maintained algorithm components to obtain the target algorithm component.

[0106] Step 203: Perform joint training using the target algorithm components.

[0107] The execution entity in this embodiment can be any one of at least two training devices participating in joint training. To achieve flexible scheduling of multiple algorithms, this application pre-constructs corresponding algorithm components based on multiple security algorithms, multiple training algorithms, and multiple model validation algorithms. Each algorithm component is used to execute its corresponding algorithm, and each training device maintains each algorithm component. It should be noted that the algorithm components may or may not be located within the training device; for example, they may be located in the cloud.

[0108] In the distributed training method provided in this application embodiment, when the training device receives target scheduling information sent by the scheduling platform, it schedules multiple pre-maintained algorithm components according to the target scheduling information to obtain a target algorithm component, and then uses the target algorithm component for joint training. The target scheduling information is generated by the scheduling platform after receiving configuration information submitted by the user, based on the initial scheduling information of at least two training devices participating in the joint training, which is included in the configuration information. Therefore, by pre-constructing multiple algorithm components and having each training device maintain these components, and by providing configuration functionality through the scheduling platform, the scheduling platform can determine the target scheduling information based on the initial scheduling information input by the user, and then send the target scheduling information to the training devices to schedule the multiple algorithm components maintained by the training devices. It is evident that this training method achieves flexible scheduling of multiple algorithms. Furthermore, since the training devices can directly perform joint training based on the scheduled target algorithm component without needing to write code based on corresponding training requirements, training efficiency is greatly improved.

[0109] To facilitate the maintenance of multiple algorithm components, in some implementations, these components can be divided into multiple component sets. Specifically, the multiple algorithm components may include multiple security components in a security component set, multiple training components in a training component set, and multiple verification components in a verification component set. Each security component executes a security algorithm, each training component executes a training algorithm, and each verification component executes a verification algorithm. Correspondingly, such as... Figure 6 As shown, step 202 may include steps 2021 and 2022.

[0110] Step 2021: If a security algorithm, a training algorithm, and a model verification algorithm are obtained from the target scheduling information, then a security component for executing the security algorithm is selected from the security component set to obtain a target security component; a component for executing the training algorithm is selected from the training component set to obtain a target training component; and a verification component for executing the model verification algorithm is selected from the verification component set to obtain a target verification component.

[0111] Specifically, a third association relationship can be pre-set between the first component identifiers of each security algorithm and each security component, a fourth association relationship between each training algorithm and each training component, and a fifth association relationship between each model verification algorithm and each verification component. Correspondingly, when the training device obtains the security algorithm, training algorithm, and model verification algorithm from the target scheduling information, it queries the associated target first component identifier in the third association relationship based on the obtained security algorithm, and determines the security component corresponding to the target first component identifier in the security component set as the target security component for executing the security algorithm; it queries the associated target second component identifier in the fourth association relationship based on the obtained training algorithm, and determines the training component corresponding to the target second component identifier in the training component set as the target training component for executing the training algorithm; and it queries the associated target third component identifier in the fifth association relationship based on the obtained model verification algorithm, and determines the verification component corresponding to the target third component identifier in the verification component set as the target verification component for executing the model verification algorithm; and the target security component, target training component, and target verification component are determined as the target algorithm component.

[0112] It is understandable that when the training device obtains the training algorithm from the target scheduling information, it can determine that the training device is the master device for joint training and execute step 2031.

[0113] Step 2022: If a security algorithm and a model verification algorithm are obtained from the target scheduling information, a security component for executing the security algorithm is selected from the set of security components to obtain a target security component; and a verification component for executing the model verification algorithm is selected from the set of verification components to obtain a target verification component.

[0114] The selection methods for target security components and target verification components can be found in the previous descriptions, and will not be repeated here.

[0115] It is understandable that when the training device does not obtain the training algorithm from the target scheduling information, it can be determined that the training device is a slave device in the joint training and step 2036 is executed.

[0116] Thus, the training device, based on the algorithms obtained from the target scheduling information, realizes the scheduling of each component in the corresponding algorithm component set.

[0117] After obtaining the target algorithm component through scheduling, joint training can be performed based on it. As mentioned earlier, in distributed training, the master and slave devices obtain different algorithms from the target scheduling information, and correspondingly, the operations performed based on the scheduled target algorithm component also differ. Specifically, for example... Figure 6 As shown, step 203 may include steps 2031 to 2039:

[0118] Step 2031: Receive the desensitized training samples sent by the slave device of the training device.

[0119] Specifically, when the training device determines that it is the master device for joint training based on the target scheduling information, it receives de-identified training samples sent by the slave device. Determining that the training device is the master device based on the target scheduling information can include: if the training device obtains the training algorithm from the target scheduling information, then the training device is determined to be the master device for joint training; or, if the label information in the sample information in the target scheduling information indicates that the first training data of the training device carries a label, then the training device is determined to be the master device. The de-identified training samples are obtained by the slave device from its second training sample through de-identification processing. The second training sample can be various types of training samples such as images, text, audio, and video. The operation process of the slave device is described in detail below; repeated details will not be repeated here.

[0120] Step 2032: Obtain multiple first training samples from the training device based on the sample information in the target scheduling information.

[0121] Specifically, sample information is obtained from the target scheduling information, and multiple first training samples for the training device are obtained based on the access information in the sample information. These first training samples can be of various types, such as images, text, audio, and video.

[0122] It is understood that the first and second training samples can vary depending on the target model being jointly trained. In one specific implementation, the target model can be a risk detection model used to detect whether a user has a default risk; correspondingly, the first and second training samples can be text-based behavioral records, which may include consumption records, repayment records, etc. In another specific implementation, the target model can be a target detection model used to detect pharmaceutical labels; correspondingly, the first and second training samples can be image data of pharmaceutical products. In yet another specific implementation, the target model can be an audio detection model; correspondingly, the first and second training samples can be audio data. These will not be listed in detail in this application; they can be set as needed in practical applications.

[0123] In some implementations, the access information in the sample information may be the access address of the database where the first training sample is located; accordingly, the training device obtains the first training sample based on the access address. In some implementations, the access information in the sample information may also be the library identifier of the database where the first training sample is located; accordingly, the training device obtains the associated access address from the sixth association relationship between the library identifier and the access address based on the library identifier, and obtains the first training sample based on the access address.

[0124] It should be noted that the execution order of steps 2031 and 2032 can be interchanged, or they can be executed simultaneously.

[0125] Step 2033: Using the target training component, desensitized training samples and the first training samples, train the first model of the training device to obtain the training result and the trained first model;

[0126] To ensure the data security of the training samples, step 2033 may include: using the second security component in the target security component to perform privacy set intersection processing on the first training sample to obtain the target first training sample; and using the target training component, the desensitized training sample and the target first training sample to train the first model of the training device to obtain the training result and the trained first model.

[0127] It should be noted that when the training mode is interactive training mode, the first model of the training device is trained using the target training component, desensitized training samples, and the target first training sample corresponding to the desensitized training sample. Since privacy set intersection processing can identify common training samples between the second and first training samples while protecting the data privacy of the master and slave devices, the target first training sample corresponding to the desensitized training sample has a corresponding relationship with the second training sample corresponding to the desensitized training sample. When the training mode is centralized training mode, the first model of the training device is also trained in each training round using the desensitized training samples and the target first training sample that have a corresponding relationship.

[0128] Step 2034: Use the first security component in the target security component to desensitize the training results to obtain desensitized training results, and send the desensitized training results to the slave device.

[0129] The processing of training results differs across different training modes. Specifically, for example... Figure 7 The diagram illustrates the interactive training mode. If the training mode in the target scheduling information is interactive training mode, then the first security component is used to de-identify the first training result of the current training round to obtain a de-identified training result, which is then sent to the slave device. Figure 8 The diagram shows a centralized training mode. If the training mode in the target scheduling information is a centralized training mode, the first security component is used to desensitize the second training results obtained after each round of training to obtain desensitized training results, and the desensitized training results are sent to the slave device.

[0130] The first and second training results may include data used to adjust model parameters. This data can be at least one of the gradient values, forward inference values, etc., from the corresponding training epochs. The second training result is the training result when the training termination condition is met. The training termination condition can be reaching a preset number of training epochs, or the loss value determined based on loss calculation being less than a preset loss value. The training termination condition can be set as needed in practical applications. The desensitized training result is the training result after desensitizing the sensitive information in the training results. The sensitive information can be data used to adjust model parameters.

[0131] In some implementations, when the training mode is interactive training mode, the master device can also send an acknowledgment information to the slave device after receiving each de-identified training sample from the slave device. This acknowledgment information may include the number of de-identified training samples, the number of features in the de-identified training samples, etc., to confirm that no data loss occurred during the transmission of the de-identified data.

[0132] Therefore, different methods are used to process the training results in different training modes, ensuring the effectiveness of the training.

[0133] Step 2035: If the training results determine that the model verification conditions are met, then the target verification component is used to verify the first model that meets the model verification conditions to obtain the first verification result.

[0134] The first verification result characterizes the accuracy of the first model. In some implementations, model verification can be performed after the target model is obtained. Accordingly, if the model verification condition is met based on the training results, it can include: if it is determined that all training samples participated in the training and the loss value determined based on the training results is less than a preset loss value, then the model verification condition is met. In some implementations, model verification can also be performed after a certain number of training epochs. Accordingly, if the model verification condition is met based on the training results, it can include: if it is determined that the training epochs corresponding to the training results have reached a preset number of epochs, then the model verification condition is met.

[0135] It is understandable that when the training device is the master device, after executing step 2021, steps 2031 to 2035 above will be executed.

[0136] Step 2036: Obtain multiple second training samples for the training device based on the sample information in the target scheduling information.

[0137] Specifically, when the training device determines that it is a slave device in the joint training based on the target scheduling information, it obtains multiple second training samples from the training device according to the sample information in the target scheduling information. The process of obtaining the second training samples is the same as the process of obtaining the first training samples, as described above; the repetitions will not be repeated here.

[0138] Step 2037: Use the first security component in the target security component to desensitize the second training sample to obtain a desensitized training sample, and send the desensitized training sample to the main device.

[0139] To ensure the data security of the second training sample, step 2037 may include: using the second security component in the target security component to perform privacy set intersection processing on the second training sample to obtain the target second training sample; and using the first security component in the target security component to perform desensitization processing on the target second training sample to obtain a desensitized training sample, and sending the desensitized training sample to the main device.

[0140] Furthermore, in order to ensure the safety of the samples and that the master and slave devices use the same training mode for joint training, the target scheduling information may also include the training mode. Accordingly, step 2037 may include: if the training mode is an interactive training mode, then each second training sample is divided to obtain multiple training sample subsets; the first security component is used to desensitize the training sample subset corresponding to the current training round to obtain desensitized training samples; if the training mode is a centralized training mode, then the first security component is used to desensitize each second training sample to obtain desensitized training samples.

[0141] Specifically, such as Figure 7 As shown, if the training mode is interactive training mode, the second security component in the target security component is used to perform privacy set intersection processing on the second training samples to obtain the target second training samples; the target second training samples are then partitioned to obtain multiple training sample subsets; the first security component is used to de-identify the training sample subset corresponding to the current training round to obtain de-identified training samples; and the de-identified training samples are sent to the main device. Figure 8 As shown, if the training mode is the centralized training mode, the second security component in the target security component is used to perform privacy set intersection processing on the second training samples to obtain the target second training samples; and the first security component in the target security component is used to perform desensitization processing on each target second training sample to obtain desensitized training samples, and the desensitized training samples are sent to the main device.

[0142] It is understandable that, such as Figure 7 As shown, when the training mode is interactive training mode, since joint training is an iterative multi-round training, in steps 1-15, if the slave device determines that the training termination condition is not met, it returns to steps 1-9 to proceed to the next round of training; in steps 1-17, if the master device determines that the training termination condition is not met, it returns to steps 1-11 to proceed to the next round of training. Figure 8 As shown, when the training mode is centralized training mode, in step 2-11, the master device performs multiple rounds of training on the first model. The first target model obtained by the master device and the second target model obtained by the slave device can be the same or different.

[0143] It should be pointed out that, Figure 7 and Figure 8 This is for illustrative purposes only and not intended to limit the scope of the instructions. The execution order of some steps can be interchanged, and they can also be performed simultaneously. For example, Figure 7 Steps 1-1 to 1-4 and steps 1-5 to 1-9 can be executed simultaneously. Figure 8Steps 2-1 to 2-4 and steps 2-5 to 2-9 can be executed simultaneously. That is, after the master device and the slave device receive the target scheduling information, they can perform the corresponding operations on their own without any order of priority. Figure 7 Steps 1-16 and 1-17 can be executed before step 1-12, or they can be executed simultaneously with step 1-12. Figure 8 Steps 2-15 can also be executed at any time during multiple rounds of training. Figure 7 and Figure 8 The specific implementation process of each step can be found in the relevant descriptions above, and the repeated parts will not be repeated here.

[0144] Step 2038: Receive the desensitization processing result sent by the master device.

[0145] Step 2039: If the parameter tuning conditions are met based on the desensitization results, then the parameters of the second model of the training device are adjusted based on the desensitization results; if the model verification conditions are met based on the desensitization results, then the second model that meets the model verification conditions is verified using the target verification component to obtain the second verification result.

[0146] Specifically, if model validation is performed after obtaining the target model, then step 2039, if it is determined that the model validation conditions are met based on the anonymization results, may include: if it is determined that the target model has been obtained based on the anonymization results, then it is determined that the model validation conditions are met. If model validation is performed after a certain number of training rounds, then step 2039, if it is determined that the model validation conditions are met based on the anonymization results, may include: if it is determined that the training rounds corresponding to the anonymization results have reached a preset number of rounds, then it is determined that the model validation conditions are met. The second validation result characterizes the accuracy of the second model.

[0147] It is understandable that when the training device is a slave device, after executing step 2022, steps 2036 to 2039 above are executed.

[0148] It is evident that regardless of whether the training device is a master or slave device, it can schedule multiple algorithm components based on the security algorithm, model verification algorithm, etc., in the target scheduling information, and perform joint training based on the scheduled target algorithm components. This not only achieves flexible scheduling of multiple algorithms but also helps to improve the training rate.

[0149] Corresponding to the aforementioned embodiments of the distributed training method, this application also provides embodiments of a distributed training apparatus. Figure 9 This is a schematic diagram illustrating the structure of a distributed training device according to an exemplary embodiment. The device can be configured in... Figure 1 In the scheduling platform shown, this device is used to execute the distributed training method applied to the scheduling platform provided in any of the above embodiments, such as... Figure 9 As shown, the distributed training device includes:

[0150] The receiving module 301 is used to receive configuration information submitted by the user. The configuration information includes the initial scheduling information of each of the at least two training devices participating in the joint training. The initial scheduling information includes partial scheduling information of various algorithm components.

[0151] The generation module 302 is used to generate target scheduling information for each training device based on the initial scheduling information of each training device; the target scheduling information includes complete scheduling information for various algorithm components.

[0152] The sending module 303 is used to send the target scheduling information of each training device to the corresponding training device. The target scheduling information is used to schedule multiple algorithm components based on the target scheduling information, and to perform joint training using the scheduled target algorithm components. Each training device maintains multiple algorithm components in advance.

[0153] In some implementations, the generation module 302 is specifically used for:

[0154] Based on the initial scheduling information of each training device, determine the common configuration information and private configuration information corresponding to each training device; the common configuration information is the same configuration information of at least two training devices, and the private configuration information is the different configuration information of the at least two training devices.

[0155] Based on the public and private configuration information of each training device, target scheduling information for each training device is generated.

[0156] In some implementations, the generation module 302 is further specifically used for:

[0157] Obtain the security algorithm, training algorithm, and model validation algorithm from the initial scheduling information.

[0158] The training mode is determined based on the security algorithm and the training algorithm; the training mode is either interactive training mode or centralized training mode.

[0159] The security algorithm, model validation algorithm, and training mode are identified as common configuration information for each training device.

[0160] In some implementations, the generation module 302 is further specifically used for:

[0161] Based on the initial scheduling information of each training device, the training algorithm and the corresponding communication address of each training device are determined.

[0162] Determine the sample information of the training samples corresponding to each training device.

[0163] For any training device among the training devices, if the training device is the master device for joint training, then the training algorithm, the sample information corresponding to the training device, and the communication address of the slave device of the training device are determined as the private configuration information corresponding to the training device.

[0164] If the training device is a slave device in joint training, then the sample information of the training device and the communication address of the master device of the training device are determined as the private configuration information corresponding to the training device.

[0165] In some embodiments, the device further includes a first acquisition module and a display module.

[0166] The first acquisition module is used to acquire device information of multiple training devices that the user has access to and algorithm information of multiple algorithms if it is determined that the login information submitted by the user to the scheduling platform is valid.

[0167] The display module is used to show the configuration interface based on device information and algorithm information. The configuration interface is used to display multiple device information and multiple algorithm information.

[0168] Correspondingly, the receiving module 301 is specifically used to receive configuration information submitted by the user based on the configuration interface.

[0169] In some embodiments, the device further includes a determining module.

[0170] The determination module is used to determine the target training device corresponding to the target training sample based on the processing information obtained from the user-submitted target training sample; the processing information includes the access information of the target training sample.

[0171] The sending module 303 is also used to send a processing request to the target training device according to the processing information; the processing request is used to request the target training device to process the target training samples according to the target structure standard to obtain standardized training samples; the target structure standard is the structure standard contained in the processing information or a preset structure standard.

[0172] In some embodiments, the device further includes a second acquisition module and a processing module.

[0173] The second acquisition module is used to acquire the user's configuration item processing request; the configuration item processing request is used to request the addition and / or deletion of the target configuration item; the target configuration item includes at least one of training device, training sample, security algorithm, training algorithm, and model verification algorithm.

[0174] The processing module is used to process requests based on configuration items and to perform corresponding processing on the target configuration items.

[0175] The distributed training device and the distributed training method provided in this application are based on the same inventive concept and have the same beneficial effects as the methods they adopt, run or implement.

[0176] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0177] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components illustrated as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0178] Corresponding to the aforementioned embodiments of the distributed training method, this application also provides an embodiment of another distributed training apparatus. Figure 10 This is a schematic diagram illustrating the structure of a distributed training device according to an exemplary embodiment. The device can be configured in... Figure 1 The training device shown is used to execute the distributed training method applied to the training device provided in any of the above embodiments, such as... Figure 10 As shown, the distributed training device includes:

[0179] The receiving module 401 is used to receive target scheduling information sent by the scheduling platform. The target scheduling information is generated by the scheduling platform after receiving the configuration information submitted by the user, based on the initial scheduling information of at least two training devices participating in joint training, which is included in the configuration information. The initial scheduling information includes partial scheduling information of various algorithm components, and the target scheduling information includes complete scheduling information of various algorithm components.

[0180] The scheduling module 402 is used to schedule multiple pre-maintained algorithm components according to the target scheduling information to obtain the target algorithm component.

[0181] Training module 403 is used for joint training using the target algorithm components.

[0182] In some implementations, the multiple algorithm components include multiple security components in a security component set, multiple training components in a training component set, and multiple verification components in a verification component set. Each security component is used to execute a security algorithm, each training component is used to execute a training algorithm, and each verification component is used to execute a verification algorithm. Correspondingly, the scheduling module 402 is specifically used for:

[0183] If a security algorithm, a training algorithm, and a model validation algorithm are obtained from the target scheduling information, then a security component for executing the security algorithm is selected from the security component set to obtain the target security component; a training component for executing the training algorithm is selected from the training component set to obtain the target training component; and a validation component for executing the model validation algorithm is selected from the validation component set to obtain the target validation component.

[0184] If a security algorithm and a model verification algorithm are obtained from the target scheduling information, then a security component for executing the security algorithm is selected from the set of security components to obtain the target security component; and a verification component for executing the model verification algorithm is selected from the set of verification components to obtain the target verification component.

[0185] In some implementations, the target scheduling information further includes sample information, which includes access information for training samples, and the target security component includes a first security component; correspondingly, the training module 403 is specifically used for:

[0186] If the training device is determined to be the master device for joint training based on the target scheduling information, then the de-identified training samples sent by the slave device of the training device are received; the de-identified training samples are obtained by the slave device from the de-identified second training sample.

[0187] Based on the access information in the sample information, obtain multiple first training samples of the training device.

[0188] The first model of the training device is trained using the target training component, desensitized training samples, and the first training samples to obtain the training results and the trained first model; the training results include data for adjusting the model parameters.

[0189] The training results are desensitized using the first security component to obtain desensitized training results, which are then sent to the slave device. The desensitized training results are the training results after desensitizing the sensitive information in the training results.

[0190] If the training results determine that the model validation conditions are met, the first model that meets the model validation conditions is validated using the target validation component to obtain the first validation result; the first validation result characterizes the accuracy of the first model.

[0191] In some implementations, the target scheduling information also includes a training mode, and the training module 403 is further specifically used for:

[0192] If the training mode is interactive training mode, the first security component is used to desensitize the first training result of the current training round to obtain the desensitized training result.

[0193] If the training mode is centralized training mode, the first security component is used to desensitize the second training results obtained after each round of training to obtain desensitized training results.

[0194] In some implementations, the training module 403 is further specifically used for:

[0195] If the target scheduling information determines that the training device is a slave device in the joint training, then multiple second training samples of the training device are obtained based on the sample information.

[0196] The first security component is used to de-identify the second training sample to obtain a de-identified training sample, which is then sent to the main device.

[0197] Receive the desensitized training results sent by the master device. The desensitized training results are the training results after the master device has desensitized the sensitive information in the training results.

[0198] If the parameter tuning conditions are met based on the desensitization training results, then the parameters of the second model of the training device are adjusted according to the desensitization training results.

[0199] If the model validation conditions are met based on the anonymization results, the second model that meets the model validation conditions is validated using the target validation component to obtain the second validation result; the second validation result characterizes the accuracy of the second model.

[0200] In some implementations, the target scheduling information also includes a training mode; correspondingly, the training module 403 is further specifically used for:

[0201] If the training mode is interactive training mode, then each second training sample is divided to obtain multiple training sample subsets; the first security component is used to desensitize the training sample subset corresponding to the current training round to obtain desensitized training samples.

[0202] If the training mode is centralized training mode, the first security component is used to desensitize each second training sample to obtain desensitized training samples.

[0203] In some implementations, the target security component further includes a second security component; the training module 403 is further specifically used for:

[0204] The first training sample is subjected to privacy set intersection processing using the second security component to obtain the target first training sample; and the first model of the training device is trained using the target training component, the desensitized training sample and the target first training sample.

[0205] The second training sample is subjected to privacy set intersection processing using the second security component to obtain the target second training sample; and the target second training sample is subjected to desensitization processing using the first security component to obtain the desensitized training sample.

[0206] The distributed training device and the distributed training method provided in this application are based on the same inventive concept and have the same beneficial effects as the methods they adopt, run or implement.

[0207] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0208] Corresponding to the aforementioned embodiments of the distributed training method, this application also provides embodiments of the distributed training system. Figure 11 This is a schematic diagram illustrating the structure of a distributed training system according to an exemplary embodiment, such as... Figure 11 As shown, the distributed training system includes a scheduling platform 501 and multiple training devices 502.

[0209] The scheduling platform 501 is used to receive configuration information submitted by the user. The configuration information includes the initial scheduling information of at least two training devices 502 participating in joint training. Based on the initial scheduling information of each of the at least two training devices 502, target scheduling information of at least two training devices 502 is generated. The target scheduling information of at least two training devices 502 is sent to the corresponding training devices 502 respectively. The initial scheduling information includes partial scheduling information of various algorithm components, and the target scheduling information includes complete scheduling information of various algorithm components.

[0210] Training device 502 is used to receive target scheduling information sent by scheduling platform 501; according to the target scheduling information, it performs scheduling processing on a variety of pre-maintained algorithm components to obtain target algorithm components; and uses the target algorithm components for joint training.

[0211] The distributed training system and the distributed training method provided in this application are based on the same inventive concept and have the same beneficial effects as the methods they adopt, run or implement.

[0212] It is understandable that the above division of modules is only a logical functional division. In actual implementation, the functions of the above modules can be integrated into hardware entities. For example, the functions of the aforementioned receiving module 301, sending module 303 and receiving module 401 can be integrated into a transceiver, and the functions of the aforementioned generating module 302, scheduling module 402 and training module 403 can be integrated into a processor, etc.

[0213] Based on this, some embodiments of this application also provide an electronic device corresponding to the distributed training method provided in the foregoing embodiments, which can be configured as follows: Figure 1 The scheduling platform or training device shown is used to execute the above-described distributed training method.

[0214] Figure 12 The present invention illustrates a hardware structure diagram of an electronic device according to an exemplary embodiment. The electronic device includes a communication interface 601, a processor 602, a memory 603, and a bus 604. The communication interface 601, processor 602, and memory 603 communicate with each other via the bus 604. The processor 602 can execute the distributed training method described above by reading and executing machine-executable instructions corresponding to the control logic of the distributed training method stored in the memory 603. The specific content of this method is described in the above embodiment and will not be repeated here.

[0215] The memory 603 mentioned in this embodiment can be any electronic, magnetic, optical, or other physical storage device, and can contain stored information such as executable instructions, data, etc. Specifically, the memory 603 can be RAM (Random Access Memory), flash memory, storage drive (such as hard disk drive), any type of storage disk (such as optical disc, DVD, etc.), or similar storage media, or combinations thereof. Communication between this system network element and at least one other network element is achieved through at least one communication interface 601 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc., can be used.

[0216] Bus 604 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 603 is used to store programs, and the processor 602 executes the programs after receiving execution instructions.

[0217] Processor 602 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 602 or by instructions in software form. The processor 602 can be a general-purpose processor, including a network processor (NP), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware controls, etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor.

[0218] The electronic device provided in this application embodiment and the distributed training method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, run or implement.

[0219] This application also provides a computer-readable storage medium corresponding to the distributed training method provided in the foregoing embodiments. Please refer to... Figure 13 As shown, the computer-readable storage medium is an optical disc 70, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the distributed training method provided in any of the foregoing embodiments.

[0220] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0221] The computer-readable storage medium provided in the above embodiments of this application and the distributed training method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0222] This application also provides a computer program product corresponding to the distributed training method provided in the foregoing embodiments. The computer program product includes a computer program that is executed by a processor to implement the distributed training method provided in the foregoing embodiments.

[0223] The computer program products provided in the above embodiments of this application and the distributed training methods provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0224] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0225] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0226] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A distributed training method, characterized in that: Applied to a scheduling platform, the method includes: receiving configuration information submitted by a user; the configuration information includes initial scheduling information of at least two training devices participating in the joint training, and the initial scheduling information includes partial scheduling information of multiple algorithm components; Determine, according to the initial scheduling information of the training device, a training algorithm and communication addresses corresponding to the training devices respectively; Determining sample information of training samples respectively corresponding to the training devices; If the training device is a master device of the joint training, determining the training algorithm, sample information corresponding to the training device, and a communication address of a slave device of the training device as private configuration information corresponding to the training device; If the training device is a slave device of the joint training, the sample information of the training device and the communication address of the master device of the training device are determined as the private configuration information corresponding to the training device; Generate target scheduling information of the training device according to the private configuration information of the training device and the initial scheduling information; the target scheduling information includes complete scheduling information of the multiple algorithm components; The target scheduling information of the training device is sent to the training device respectively, the target scheduling information is used to schedule the multiple algorithm components based on the target scheduling information, and joint training is performed based on the scheduled target algorithm components. The training device maintains the multiple algorithm components in advance.

2. The method according to claim 1, characterized in that: The step of generating target scheduling information of the training device according to the private configuration information of the training device and the initial scheduling information includes: Determine, according to the initial scheduling information of the training device, the common configuration information corresponding to the training device; the common configuration information is the same configuration information of the at least two training devices; The target scheduling information of the training device is generated according to the public configuration information and the private configuration information corresponding to the training device.

3. The method according to claim 2, characterized in that The determining, according to the initial scheduling information of the training device, the public configuration information corresponding to the training device comprises: Obtaining a security algorithm, a training algorithm, and a model verification algorithm from the initial scheduling information; Determine a training mode according to the safety algorithm and the training algorithm; the training mode includes an interactive training mode or a centralized training mode; The security algorithm, the model verification algorithm and the training mode are determined as public configuration information corresponding to the training device.

4. The method according to any one of claims 1 to 3, characterized in that: Before receiving the configuration information submitted by the user, the method further includes: If it is determined that the login information submitted by the user to the scheduling platform is valid, obtaining device information of the training device and algorithm information of the algorithm to which the user has access rights; Display a configuration interface according to the device information and the algorithm information, wherein the configuration interface is used to display the device information and the algorithm information; The receiving of configuration information submitted by the user includes: Receive configuration information submitted by the user based on the configuration interface.

5. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: If processing information of the target training sample submitted by the user is obtained, the target training device corresponding to the target training sample is determined according to the processing information; the processing information includes access information of the target training sample; According to the processing information, a processing request is sent to the target training device; the processing request is used to request the target training device to process the target training sample according to the target structure standard to obtain a standardized training sample; the target structure standard is the structure standard contained in the processing information, or a preset structure standard.

6. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Obtaining a configuration item processing request from a user; the configuration item processing request is used to request adding and / or deleting a target configuration item; the target configuration item includes at least one of a training device, a training sample, a security algorithm, a training algorithm, and a model verification algorithm; According to the configuration item processing request, the target configuration item is processed accordingly.

7. A distributed training method, characterized in that: Applied to a training device, the method comprises: Receive target scheduling information sent by a scheduling platform; the target scheduling information is generated by the scheduling platform after receiving configuration information submitted by a user, based on the initial scheduling information of at least two training devices participating in the joint training included in the configuration information; the initial scheduling information includes partial scheduling information of multiple algorithm components, the target scheduling information includes complete scheduling information of the multiple algorithm components, the complete scheduling information includes private configuration information, and when the training device is a master device of the joint training, the private configuration information includes the training algorithm, sample information corresponding to the training device, and a communication address of a slave device of the training device; when the training device is a slave device of the joint training, the private configuration information includes the sample information of the training device and the communication address of the master device of the training device; According to the target scheduling information, scheduling processing is performed on the plurality of algorithm components maintained in advance to obtain a target algorithm component; The target algorithm components are used for joint training.

8. The method according to claim 7, characterized in that The multiple algorithm components include multiple security components in the security component set, multiple training components in the training component set, and multiple verification components in the verification component set, each security component is used to execute a security algorithm, each training component is used to execute a training algorithm, and each verification component is used to execute a verification algorithm; The step of performing scheduling processing on the plurality of algorithm components maintained in advance according to the target scheduling information to obtain the target algorithm component includes: If a security algorithm, a training algorithm and a model verification algorithm are obtained from the target scheduling information, a security component for executing the security algorithm is selected from the security component set to obtain a target security component; a training component for executing the training algorithm is selected from the training component set to obtain a target training component; and selecting a verification component for executing the model verification algorithm from the verification component set to obtain a target verification component; If a security algorithm and a model verification algorithm are obtained from the target scheduling information, a component for executing the security algorithm is selected from the security component set to obtain a target security component; and a component for executing the model verification algorithm is selected from the verification component set to obtain a target verification component.

9. The method according to claim 8, characterized in that The sample information includes access information of the training sample, the target security component includes a first security component; and the joint training using the target algorithm component includes: If it is determined according to the target scheduling information that the training device is the master device of the joint training, receiving a desensitized training sample sent by a slave device of the training device; the desensitized training sample is obtained by the slave device performing desensitization processing on the second training sample; Acquire a plurality of first training samples of the training device according to the access information; Using the target training component, the desensitized training sample and the first training sample, the first model of the training device is trained to obtain a training result and a trained first model; the training result includes data for adjusting model parameters; Desensitizing the training result by using the first security component to obtain a desensitized training result, and sending the desensitized training result to the slave device; the desensitized training result is a training result after desensitizing sensitive information in the training result; If it is determined according to the training result that the model verification condition is met, the target verification component is used to verify the first model when the model verification condition is met to obtain a first verification result; the first verification result represents the accuracy of the first model.

10. The method according to claim 9, characterized in that The target scheduling information also includes a training mode; and the desensitizing the training result by using the first security component to obtain a desensitized training result includes: If the training mode is an interactive training mode, desensitizing the first training result of the current training round by using the first security component to obtain a desensitized training result; If the training mode is a centralized training mode, the first security component is used to perform desensitization processing on the second training results obtained after each round of training to obtain a desensitized training result.

11. The method according to claim 9, characterized in that The joint training using the target algorithm component also includes: If it is determined according to the target scheduling information that the training device is a slave device of the joint training, acquiring a plurality of second training samples of the training device according to the sample information; Using the first security component to perform desensitization processing on the second training sample to obtain a desensitized training sample, and sending the obtained desensitized training sample to a master device of the training device; Receive a desensitization training result sent by the master device; the desensitization training result is a training result after the master device desensitizes sensitive information in the training result; If it is determined that the parameter adjustment condition is met according to the desensitization training result sent by the master device, parameter adjustment processing is performed on the second model of the training device according to the desensitization training result sent by the master device; If it is determined that the model verification conditions are met based on the desensitization training results sent by the main device, the target verification component is used to verify the second model when the model verification conditions are met to obtain a second verification result; the second verification result represents the accuracy of the second model.

12. The method according to claim 11, characterized in that The target scheduling information also includes a training mode, and the desensitizing the second training sample using the first security component to obtain a desensitized training sample includes: If the training mode is an interactive training mode, each of the second training samples is divided and processed to obtain a plurality of training sample subsets; the first security component is used to perform desensitization processing on the training sample subset corresponding to the current training round to obtain a desensitized training sample; If the training mode is a centralized training mode, the first security component is used to perform desensitization processing on each of the second training samples to obtain a desensitized training sample.

13. The method according to claim 11, characterized in that The target security component also includes a second security component; and the training of the first model of the training device using the target training component, the desensitized training sample and the first training sample includes: Using the second security component to perform privacy set intersection processing on the first training sample to obtain a target first training sample; using the target training component, the desensitized training sample and the target first training sample to train the first model of the training device; the target first training sample is a training sample shared by the first training sample and the second training sample of the slave device; The using the first security component to perform desensitization processing on the second training sample to obtain a desensitized training sample includes: The second security component is used to perform the privacy set intersection processing on the second training sample to obtain a target second training sample; and the first security component is used to perform desensitization processing on the target second training sample to obtain a desensitized training sample; the target second training sample is a training sample common to the second training sample and the first training sample of the main device.

14. A distributed training device, characterized in that: Applied to a dispatching platform, the device comprises: A receiving module, configured to receive configuration information submitted by a user; the configuration information includes initial scheduling information of at least two training devices participating in the joint training, and the initial scheduling information includes partial scheduling information of multiple algorithm components; A generation module, used to determine the training algorithm and the communication address corresponding to the training device according to the initial scheduling information of the training device; determine the sample information of the training samples corresponding to the training device; if the training device is the master device of the joint training, determine the training algorithm, the sample information corresponding to the training device, and the communication address of the slave device of the training device as the private configuration information corresponding to the training device; if the training device is the slave device of the joint training, determine the sample information of the training device and the communication address of the master device of the training device as the private configuration information corresponding to the training device; generate the target scheduling information of the training device according to the private configuration information of the training device and the initial scheduling information; the target scheduling information includes the complete scheduling information of the multiple algorithm components; A sending module is used to send the target scheduling information of the training device to the training device respectively, the target scheduling information is used to schedule the multiple algorithm components based on the target scheduling information, and perform joint training based on the scheduled target algorithm components, and the training device pre-maintains the multiple algorithm components.

15. A distributed training device, characterized in that: Applied to training equipment, the device comprises: A receiving module, used to receive target scheduling information sent by a scheduling platform; the target scheduling information is generated by the scheduling platform after receiving the configuration information submitted by the user, based on the initial scheduling information of at least two training devices participating in the joint training included in the configuration information; the initial scheduling information includes partial scheduling information of multiple algorithm components, the target scheduling information includes complete scheduling information of the multiple algorithm components, and the complete scheduling information includes private configuration information. When the training device is a master device of the joint training, the private configuration information includes the training algorithm, sample information corresponding to the training device, and a communication address of a slave device of the training device; when the training device is a slave device of the joint training, the private configuration information includes the sample information of the training device and the communication address of the master device of the training device; A scheduling module, configured to perform scheduling processing on the plurality of algorithm components maintained in advance according to the target scheduling information to obtain a target algorithm component; A training module is used to perform joint training using the target algorithm component.

16. A distributed training system, characterized in that: Includes: a dispatching platform and multiple training devices; The scheduling platform is used to receive configuration information submitted by a user, wherein the configuration information includes initial scheduling information of at least two training devices participating in the joint training among the multiple training devices; Determine the training algorithm and the communication addresses corresponding to the training devices respectively according to the initial scheduling information of the training devices; determine the sample information of the training samples corresponding to the training devices respectively; If the training device is the master device of the joint training, the training algorithm, the sample information corresponding to the training device, and the communication address of the slave device of the training device are determined as the private configuration information corresponding to the training device; if the training device is the slave device of the joint training, the sample information of the training device and the communication address of the master device of the training device are determined as the private configuration information corresponding to the training device; according to the private configuration information of the training device and the initial scheduling information, the target scheduling information of the training device is generated; sending target scheduling information of the training devices to the training devices respectively; The initial scheduling information includes partial scheduling information of multiple algorithm components, and the target scheduling information includes complete scheduling information of the multiple algorithm components; The training device is used to receive the target scheduling information sent by the scheduling platform; schedule the multiple algorithm components maintained in advance according to the target scheduling information to obtain the target algorithm component; and use the target algorithm component to perform joint training.

17. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processor executes the computer program to implement the method according to any one of claims 1 to 6, or to implement the method according to any one of claims 7 to 13.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the method according to any one of claims 1 to 6, or to implement the method according to any one of claims 7 to 13.

19. A computer program product comprising a computer program, characterized in that The computer program is executed by a processor to implement the method according to any one of claims 1 to 6, or to implement the method according to any one of claims 7 to 13.

Images