Method and apparatus for determining training data

Abstract

The application discloses a method and device for determining training data, and relates to the technical field of computers. The method comprises the following steps: acquiring a training data set; for each training party, based on the identification of the data, the data is cyclically distributed to each computing node in the training party; based on the preset data amount and the total number of computing nodes contained in the training party, the data amount of each computing node in the training party to acquire data in each round of training is determined; for each computing node in each training party, based on the preset data amount, the total number of computing nodes contained in the training party and the data index identification adopted by the computing node in the last round of training, the data index identification adopted by the computing node in the current round of training is determined; based on the determined data amount and the adopted data index identification, the training data used by the node in the current round of training is determined. The method can ensure the accuracy of vertical federated learning.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to methods and apparatus for determining training data. Background Technology

[0002] Distributed deep learning models enable data-parallel distributed training, meaning that deep learning models are trained simultaneously on different computing nodes using different data, thereby improving training speed and the accuracy of the trained model. In existing distributed deep learning systems, training data distribution methods include: uniformly dividing the training data into multiple parts and distributing each part to various computing nodes, or having each computing node retrieve data from a network server during training.

[0003] However, when performing vertical federated training, the data distribution method of evenly dividing the training data in the dataset into multiple parts and allocating each part of the data to each computing node in the vertical federated training can lead to model training failure. The data distribution method in which each computing node obtains data from the network server during the training process wastes communication resources and increases training time. Summary of the Invention

[0004] This disclosure provides a method, apparatus, electronic device, and computer-readable storage medium for determining training data.

[0005] According to a first aspect of this disclosure, a method for determining training data is provided, comprising: acquiring a training dataset; for each of at least two training parties participating in longitudinal federated training, cyclically distributing data in the training dataset to each computing node in that training party based on a data identifier; for each training party, determining the amount of data acquired by each computing node in that training party in each training round based on a preset data volume and the total number of computing nodes included in that training party; if the current round is not the first round of training, for each computing node in that training party, determining the data index identifier used by that computing node in the current round of training based on the preset data volume, the total number of computing nodes included in that training party, and the data index identifier used by that computing node in the previous round of training; and determining the training data used by that node in the current round of training based on the determined amount of data acquired by that computing node in each round of training and the data index identifier used by that computing node in the current round of training.

[0006] In some embodiments, for each of the at least two trainers participating in vertical federated training, data in the training dataset is cyclically distributed to each computing node in that trainer based on the data identifier, including: for each of the at least two trainers participating in vertical federated training, determining each computing node in that trainer; and cyclically distributing each piece of data in the training dataset to each computing node based on the data identifier.

[0007] In some embodiments, determining the amount of data acquired by each computing node in the training process in each round of training, based on a preset amount of data and the total number of computing nodes included in the training process, includes: determining the amount of data acquired by each computing node in the training process in each round of training based on the quotient between the preset amount and the total number of computing nodes included in the training process.

[0008] In some embodiments, if the current round is not the first round of training, for each computing node in each training side, based on a preset amount of data, the total number of computing nodes in the training side, and the data index identifier used by the computing node in the previous round of training, the data index identifier used by the computing node in the current round of training is determined, including: if the current round is not the first round of training, for each computing node in each training side, a first remainder is obtained by taking the sum of a first difference and the total number of computing nodes in the training side, and then dividing the remainder by the total number of computing nodes. The first remainder is determined as the data index identifier used by the computing node in the current round of training, wherein the first difference is the difference between the data index identifier used by the computing node in the previous round of training and the second remainder, and the second remainder is the result obtained by dividing the preset amount by the total number of computing nodes.

[0009] In some embodiments, the method for determining training data further includes: determining whether the current training round is the first training round; if the current round is the first training round, for each computing node in each training party, using the node index identifier of the computing node as the data index identifier used by the computing node in the current training round.

[0010] In some embodiments, after determining the data used by each computing node in each training party in the current training round, the method for determining training data further includes: in the current round of longitudinal federated training, using the data used by the computing node in the current round of training to participate in longitudinal federated training.

[0011] According to a second aspect of this disclosure, an apparatus for determining training data is provided, comprising: an acquisition unit configured to acquire a training dataset; a distribution unit configured to, for each of at least two training parties participating in longitudinal federated training, cyclically distribute data in the training dataset to each computing node in that training party based on a data identifier; a first determining unit configured to, for each training party, determine the amount of data acquired by each computing node in that training party in each training round based on a preset data amount and the total number of computing nodes included in that training party; a second determining unit configured to, if the current round is not the first round of training, determine the data index identifier used by the computing node in the current round of training for each computing node in that training party based on the preset data amount, the total number of computing nodes included in that training party, and the data index identifier used by the computing node in the previous round of training; and a third determining unit configured to, based on the determined amount of data acquired by the computing node in each round of training and the data index identifier used by the computing node in the current round of training, determine the training data used by the node in the current round of training.

[0012] In some embodiments, the distribution unit includes: a first determining module configured to determine each computing node in each of at least two training parties participating in longitudinal federated training; and a distribution module configured to distribute each piece of data in the training dataset to each computing node in a cyclic manner based on the data identifier.

[0013] In some embodiments, the first determining unit includes: a second determining module configured to determine the amount of data acquired by each computing node in the training party in each round of training based on a quotient between a preset number and the total number of computing nodes included in the training party.

[0014] In some embodiments, the second determining unit includes: a third determining module, configured to, if the current round is not the first round of training, for each computing node in each training party, take the sum of a first difference and the total number of computing nodes in the training party, and then take the remainder after dividing by the total number of computing nodes to obtain a first remainder, and determine the first remainder as the data index identifier used by the computing node in the current round of training, wherein the first difference is the difference between the data index identifier used by the computing node in the previous round of training and the second remainder, and the second remainder is the result obtained by taking the remainder after dividing the preset number by the total number of computing nodes.

[0015] In some embodiments, the apparatus for determining training data further includes: a fourth determining unit, configured to determine whether the current round of training is the first round of training; if the current round is the first round of training, for each computing node in each training party, the node index identifier of the computing node is used as the data index identifier used by the computing node in the current round of training.

[0016] In some embodiments, the means for determining training data further includes: a training unit configured to participate in longitudinal federated training using the data used by the computing node in the current round of training.

[0017] According to a third aspect of this disclosure, embodiments of this disclosure provide an electronic device including: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining training data as provided in the first aspect.

[0018] According to a fourth aspect of this disclosure, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method for determining training data as provided in the first aspect.

[0019] The method and apparatus for determining training data provided in this disclosure include: acquiring a training dataset; for each of at least two training parties participating in vertical federated training, cyclically distributing data from the training dataset to each computing node in that training party based on data identifiers; for each training party, determining the amount of data acquired by each computing node in each training round based on a preset data volume and the total number of computing nodes included in that training party; if the current round is not the first round of training, for each computing node in each training party, determining the data index identifier used by that computing node in the current round of training based on the preset data volume, the total number of computing nodes included in that training party, and the data index identifier used by that computing node in the previous round of training; and determining the training data used by that node in the current round of training based on the determined amount of data acquired by that computing node in each round of training and the data index identifier used by that computing node in the current round of training. This method can ensure the accuracy of vertical federated learning while saving communication resources and training time.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0021] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of this application. Wherein:

[0022] Figure 1 This is an exemplary system architecture diagram in which embodiments of this application can be applied;

[0023] Figure 2 This is a flowchart of an embodiment of the method for determining training data according to this application;

[0024] Figure 3 This is a schematic diagram illustrating the distribution of data in one embodiment of the method for determining training data according to this application;

[0025] Figure 4 This is a schematic diagram illustrating the distribution of data in one embodiment of the method for determining training data according to this application;

[0026] Figure 5 This is a flowchart of another embodiment of the method for determining training data according to this application;

[0027] Figure 6 This is a schematic diagram of one embodiment of the apparatus for determining training data according to this application;

[0028] Figure 7 This is a block diagram of an electronic device used to implement the method for determining training data in the embodiments of this application. Detailed Implementation

[0029] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0030] Figure 1 An exemplary system architecture 100 is shown, in which embodiments of the method or apparatus for determining training data of this application may be applied.

[0031] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0032] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be user terminal devices, on which various client applications can be installed, such as data maintenance applications, image applications, video applications, search applications, financial applications, etc.

[0033] Terminal devices 101, 102, and 103 can be various electronic devices with displays that support receiving messages from the server, including but not limited to smartphones, tablets, e-book readers, electronic players, laptops, and desktop computers.

[0034] Terminal devices 101, 102, and 103 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules (e.g., multiple software modules used to provide distributed services) or as a single software program or software module. No specific limitations are made here.

[0035] Server 105 can acquire the training dataset. For each of the at least two training parties participating in vertical federated training, based on the data identifier, the data in the training dataset is cyclically distributed to each computing node in that training party. For each training party, based on the preset data volume and the total number of computing nodes in the training party, the amount of data acquired by each computing node in that training party in each training round is determined. If the current round is not the first round of training, for each computing node in each training party, based on the preset data volume, the total number of computing nodes in that training party, and the data index identifier used by the computing node in the previous round of training, the data index identifier used by the computing node in the current round of training is determined. Based on the determined amount of data acquired by the computing node in each round of training and the data index identifier used by the computing node in the current round of training, the training data used by the node in the current round of training is determined.

[0036] It should be noted that the method for determining training data provided in the embodiments of this disclosure can be executed by server 105, and correspondingly, the device for determining training data can be set in server 105.

[0037] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0038] Continue to refer to Figure 2 The flowchart 200 of an embodiment of a method for determining training data according to the present disclosure is shown, including the following steps:

[0039] Step 201: Obtain the training dataset.

[0040] In this embodiment, the execution entity of the method for determining training data (e.g.) Figure 1 The server shown can obtain the training dataset via wired or wireless means. The training dataset includes multiple data points used for longitudinal federated training.

[0041] Step 202: For each of the at least two trainers participating in the vertical federated training, based on the data identifier, the data in the training dataset is cyclically distributed to each computing node in that trainer.

[0042] In this embodiment, for each of the at least two training parties participating in vertical federated training, the data in the training dataset can be cyclically distributed to each computing node of that training party based on the data identifier. The training data of different training parties participating in vertical federated training have the same identifier (i.e., the identifiers are aligned) but different features (i.e., each party has its own private data features). For example, the private data of organization A participating in vertical federated training is the browsing data of item 1, item 2, and item 3, and the private data of organization B participating in vertical federated training is the purchase data of item 1, item 2, and item 3.

[0043] Distributing data from the training dataset cyclically to each computation node in the training dataset based on the data's identifier means, for example... Figure 3 As shown, the data belonging to the training party, or the private data owned by the training party, is distributed cyclically to each computing node in the training party based on the data identifier. It should be noted that the data indicated by identifier 1 owned by training party A and the data indicated by identifier 1 owned by training party B only have the same data identifier (e.g., data belonging to the same user), but the characteristics of the indicated data are different (e.g., data used to describe different characteristics of the same user).

[0044] Step 203: For each training side, based on the preset data volume and the total number of computing nodes included in the training side, determine the amount of data that each computing node in the training side will acquire in each round of training.

[0045] In this embodiment, for each of the at least two training parties participating in vertical federated training, the amount of training data that each computing node in the training party should use and acquire in each round of vertical federated training can be determined based on a preset number and the total number of computing nodes contained in the training party.

[0046] Step 204: If the current round is not the first round of training, for each computing node in each training side, based on the preset data volume, the total number of computing nodes in the training side, and the data index identifier used by the computing node in the previous round of training, determine the data index identifier used by the computing node in the current round of training.

[0047] In this embodiment, if the current round is not the first round of training, for each computing node in each training side, based on the preset data volume, the total number of computing nodes in the training side, and the data index identifier used by the computing node in the previous round of training, the data index identifier used by the computing node in the current round of training is determined, so as to ensure that the identifiers of the training data used by training sides with different numbers of computing nodes in the same round of training are aligned (consistent).

[0048] Step 205: Based on the amount of data acquired by the computing node in each training round and the data index identifier used by the computing node in the current training round, determine the training data used by the node in the current training round.

[0049] In this embodiment, for each computing node in each training party, the training data used by that node in the current training round can be determined based on the amount of data acquired by that computing node in each training round and the data index identifier used by that computing node in the current training round. Specifically, the data is retrieved from the data allocated to that computing node using the data index identifier of that computing node in the current training round, and the data obtained is taken from the retrieved data, with the amount equal to the amount of data acquired by that computing node in each training round. This retrieved data is then determined as the data used by that computing node in the current training round, that is, the training data used by that computing node when participating in the current vertical federated training.

[0050] The method for determining training data provided in this embodiment obtains a training dataset; for each of at least two training parties participating in vertical federated training, based on the data identifier, the data in the training dataset is cyclically distributed to each computing node in that training party; for each training party, based on a preset data volume and the total number of computing nodes included in that training party, the amount of data acquired by each computing node in that training party in each round of training is determined; if the current round is not the first round of training, for each computing node in each training party, based on the preset data volume, the total number of computing nodes included in that training party, and the data index identifier used by that computing node in the previous round of training, the data index identifier used by that computing node in the current round of training is determined; based on the determined amount of data acquired by that computing node in each round of training and the data index identifier used by that computing node in the current round of training, the training data used by that node in the current round of training is determined. This ensures that in distributed vertical federated deep learning, the identifiers of the training data used by each training party participating in the vertical federated training remain aligned (consistent) in each round of training, ensuring the accuracy of vertical federated learning.

[0051] Furthermore, during the training process, each computing node obtains the training data for each round from its own local machine (the trainer), avoiding requests for data from the network server, thus improving training efficiency and saving network transmission resources.

[0052] Optionally, for each of the at least two trainers participating in the vertical federated training, the data in the training dataset is cyclically distributed to each computing node in that trainer, including: for each of the at least two trainers participating in the vertical federated training, determining each computing node in that trainer; and, based on the data identifier, sequentially distributing each piece of data in the training dataset to each computing node in a cyclical manner.

[0053] In this embodiment, for each of the at least two training parties participating in vertical federated training, each computing node contained in the training party is first determined. Then, each piece of data in the training dataset belonging to the training party is distributed to each computing node in a cyclical manner according to the order of the computing nodes contained in the training party.

[0054] like Figure 4 As shown in the figure, for a certain training method, each piece of data (data 1 to data k) belonging to that training method in the training dataset can be distributed to each computing node in a cyclical manner according to the order of the computing nodes. Here, m represents the total number of computing nodes, i represents the identifier / position of the computing node, and n represents the counting identifier.

[0055] Optionally, based on a preset amount of data and the total number of computing nodes included in the training matrix, the amount of data acquired by each computing node in the training matrix in each training round is determined, including: determining the amount of data acquired by each computing node in the training matrix in each training round based on the quotient between the preset amount and the total number of computing nodes included in the training matrix.

[0056] In this embodiment, the amount of data that each computing node in the training party should acquire in each round of training can be determined based on the quotient between a preset number and the total number of computing nodes included in the training party. Based on the node's position or identifier, the amount of training data that the computing node should use and acquire in each round of vertical federated training is determined, specifically as follows:

[0057]

[0058] Among them, b i B represents the amount of training data that the computing node identified as i in the training party should use and acquire in each round of vertical federated training; B represents the amount of data in a small batch of data set (Batch) in each round of deep learning training; m represents the total number of computing nodes owned by the training party.

[0059] Optionally, if the current round is not the first round of training, for each computing node in each training side, based on the preset data volume, the total number of computing nodes in the training side, and the data index identifier used by the computing node in the previous round of training, the data index identifier used by the computing node in the current round of training is determined, including: if the current round is not the first round of training, for each computing node in each training side, a first remainder is obtained by taking the sum of a first difference and the total number of computing nodes in the training side, and then dividing the remainder by the total number of computing nodes. The first remainder is determined as the data index identifier used by the computing node in the current round of training, wherein the first difference is the difference between the data index identifier used by the computing node in the previous round of training and the second remainder, and the second remainder is the result obtained by dividing the preset number by the total number of computing nodes.

[0060] In this embodiment, if the current round is not the first round of training, for each computing node in each training side, the first remainder is obtained by taking the sum of the first difference and the total number of computing nodes in the training side, modulo the total number of computing nodes in the training side. This first remainder is determined as the data index identifier of the computing node in the current round of training. The first difference is the difference between the data index identifier used by the computing node in the previous round of training and the second remainder. The second remainder is the result obtained by taking a preset number modulo the total number of computing nodes in the current training side. Specifically:

[0061]

[0062] in, This represents the data index identifier of the computing node in the j-th round of training; B represents the data index identifier used by the computing node in the previous training round of round j; B represents the data size of a small batch of data in each round of deep learning training; m represents the total number of computing nodes owned by the training party.

[0063] It should be noted that, in the actual training process, data points with a remaining quantity of less than B from all k data points are discarded.

[0064] Continue to refer to Figure 5 The flowchart 500 of another embodiment of the method for determining training data according to the present disclosure is shown, including the following steps:

[0065] Step 501: Obtain the training dataset.

[0066] Step 502: For each of the at least two trainers participating in the longitudinal federated training, based on the data identifier, the data in the training dataset is cyclically distributed to each computing node in that trainer.

[0067] Step 503: For each training side, based on the preset data volume and the total number of computing nodes included in the training side, determine the amount of data that each computing node in the training side will acquire in each round of training.

[0068] Step 504: Determine whether the current training round is the first round of training. If the current round is the first round of training, for each computing node in each training party, use the node index identifier of the computing node as the data index identifier used by the computing node in the current training round.

[0069] In this embodiment, it is determined whether the current training round is the first round of training. If the current round is the first round of training, for each computing node in each of the at least two training parties, the node index identifier of the computing node is used as the data index identifier of the computing node in the current training round (i.e., the first round of training).

[0070] Specifically, in the first round of training, .

[0071] Step 505: If the current round is not the first round of training, for each computing node in each training side, based on the preset data volume, the total number of computing nodes in the training side, and the data index identifier used by the computing node in the previous round of training, determine the data index identifier used by the computing node in the current round of training.

[0072] Step 506: Based on the amount of data acquired by the computing node in each training round and the data index identifier used by the computing node in the current training round, determine the training data used by the node in the current training round.

[0073] In this embodiment, the descriptions of steps 501, 502, 503, 505, and 506 are consistent with the descriptions of steps 201, 202, 203, 204, and 205, and will not be repeated here.

[0074] The method for determining training data provided in this embodiment is compared to... Figure 2 The method provided by the described embodiments clarifies how to determine the data index identifier used by the computing node in the first round of training, thereby clarifying how each computing node determines the training data in the first round of training and improving the efficiency of determining the training data.

[0075] In the above combination Figure 2 and Figure 5 In some optional implementations of the described embodiments, after determining the data used by each computing node in each training party in the current training round, the method for determining the training data further includes: in the current round of longitudinal federated training, using the data used by the computing node in the current round of training to participate in the longitudinal federated training.

[0076] In this embodiment, for each computing node in each training party, after determining the data used by the computing node in the current round of training, the determined data is used as training data. Based on the computing node participating in vertical federated training, it can be ensured that the identifiers of the training data used by each party are aligned in each round of training, which can ensure the effectiveness and accuracy of vertical federated training.

[0077] Further reference Figure 6 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a device for determining training data, which is similar to... Figure 2 and Figure 5 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0078] like Figure 6 As shown, the apparatus for determining training data in this embodiment includes: an acquisition unit 601, a distribution unit 602, a first determination unit 603, a second determination unit 604, and a third determination unit 605. The system includes the following components: an acquisition unit, configured to acquire a training dataset; a distribution unit, configured to, for each of at least two training parties participating in longitudinal federated training, cyclically distribute data from the training dataset to each computing node within that training party based on data identifiers; a first determination unit, configured to, for each training party, determine the amount of data acquired by each computing node in each training round based on a preset data volume and the total number of computing nodes included in that training party; a second determination unit, configured to, if the current round is not the first round of training, determine the data index identifier used by each computing node in each training party in the current round of training based on a preset data volume, the total number of computing nodes included in that training party, and the data index identifier used by the computing node in the previous round of training; and a third determination unit, configured to, based on the determined amount of data acquired by the computing node in each round of training and the data index identifier used by the computing node in the current round of training, determine the training data used by the node in the current round of training.

[0079] In some embodiments, the distribution unit includes: a first determining module configured to determine each computing node in each of at least two training parties participating in longitudinal federated training; and a distribution module configured to distribute each piece of data in the training dataset to each computing node in a cyclic manner based on the data identifier.

[0080] In some embodiments, the first determining unit includes: a second determining module configured to determine the amount of data acquired by each computing node in the training party in each round of training based on a quotient between a preset number and the total number of computing nodes included in the training party.

[0081] In some embodiments, the second determining unit includes: a third determining module, configured to, if the current round is not the first round of training, for each computing node in each training party, take the sum of a first difference and the total number of computing nodes in the training party, and then take the remainder after dividing by the total number of computing nodes to obtain a first remainder, and determine the first remainder as the data index identifier used by the computing node in the current round of training, wherein the first difference is the difference between the data index identifier used by the computing node in the previous round of training and the second remainder, and the second remainder is the result obtained by taking the remainder after dividing the preset number by the total number of computing nodes.

[0082] In some embodiments, the apparatus for determining training data further includes: a fourth determining unit, configured to determine whether the current round of training is the first round of training; if the current round is the first round of training, for each computing node in each training party, the node index identifier of the computing node is used as the data index identifier used by the computing node in the current round of training.

[0083] In some embodiments, the means for determining training data further includes: a training unit configured to participate in longitudinal federated training using the data used by the computing node in the current round of training.

[0084] Each unit in the aforementioned device 600 and the reference Figure 2 Figure 5 The steps described in the method correspond to those steps. Therefore, the operations, features, and technical effects described above for the method used to determine the training data also apply to device 600 and the units contained therein, and will not be repeated here.

[0085] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0086] like Figure 7 The diagram shown is a block diagram of an electronic device 700 for determining training data according to an embodiment of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0087] like Figure 7As shown, the electronic device includes one or more processors 701, a memory 702, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 7 Take the 701 processor as an example.

[0088] The memory 702 is the non-transitory computer-readable storage medium provided in this application. This memory stores instructions executable by at least one processor to cause the at least one processor to perform the method for determining training data provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the method for determining training data provided in this application.

[0089] Memory 702, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the method for determining training data in the embodiments of this application (e.g., appendix). Figure 6 The acquisition unit 601, distribution unit 602, first determination unit 603, second determination unit 604, and third determination unit 605 are shown. The processor 701 executes various functional applications and data processing of the server by running non-transient software programs, instructions, and modules stored in the memory 702, that is, it implements the method for determining training data in the above method embodiments.

[0090] Memory 702 may include a program storage area and a data storage area. The program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device for extracting video clips. Furthermore, memory 702 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 702 may optionally include memory remotely located relative to processor 701, and this remote memory may be connected to the electronic device for extracting video clips via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0091] The electronic device used for determining the training data may further include an input device 703, an output device 704, and a bus 705. The processor 701, memory 702, input device 703, and output device 704 may be connected via the bus 705 or other means. Figure 7 Taking the connection between China and Israel via bus 705 as an example.

[0092] Input device 703 can receive input digital or character information, as well as generate key signal inputs related to user settings and function control of the electronic device used to extract video clips, such as touch screens, keypads, mice, trackpads, touchpads, joysticks, one or more mouse buttons, trackballs, joysticks, etc. Output device 704 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The display device may include, but is not limited to, liquid crystal displays (LCDs), light-emitting diode (LED) displays, and plasma displays. In some embodiments, the display device may be a touch screen.

[0093] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0094] These computational programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0095] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0096] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0097] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0098] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0099] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for determining training data, comprising: Obtain the training dataset; For each of the at least two trainers participating in the vertical federated training, the data in the training dataset is cyclically distributed to each computing node in that trainer based on the data identifier. For each training side, based on the preset data volume and the total number of computing nodes included in the training side, the amount of data acquired by each computing node in the training side in each round of training is determined. If the current round is not the first round of training, for each computing node in each training side, based on the preset data volume, the total number of computing nodes in the training side, and the data index identifier used by the computing node in the previous round of training, the data index identifier used by the computing node in the current round of training is determined. Based on the amount of data acquired by the computing node in each training round and the data index identifier used by the computing node in the current training round, the training data used by the node in the current training round is determined.

2. The method of claim 1, wherein, The step of cyclically distributing data from the training dataset to each computing node within at least two training parties participating in vertical federated training, based on data identifiers, includes: For each of the at least two trainers participating in the vertical federated training, determine each computing node of that trainer. Based on the identifier of the data, each piece of data in the training dataset is sequentially distributed to each computing node in a cyclical manner.

3. The method of claim 1, wherein, The determination of the amount of data acquired by each computing node in each round of training, based on a preset data volume and the total number of computing nodes included in the training matrix, includes: Based on the quotient between the preset data volume and the total number of computing nodes included in the training method, the amount of data acquired by each computing node in the training method in each round of training is determined.

4. The method of claim 1, wherein, If the current round is not the first round of training, for each computing node in each training party, based on the preset data volume, the total number of computing nodes in the training party, and the data index identifier used by the computing node in the previous round of training, the data index identifier used by the computing node in the current round of training is determined, including: If the current round is not the first round of training, for each computing node in each training side, a first remainder is obtained by taking the sum of the first difference and the total number of computing nodes in the training side, modulo the total number of computing nodes. The first remainder is determined as the data index identifier used by the computing node in the current round of training. The first difference is the difference between the data index identifier used by the computing node in the previous round of training and the second remainder. The second remainder is the result obtained by taking the preset data amount modulo the total number of computing nodes.

5. The method of claim 1, wherein, The method further includes: Determine whether the current training round is the first round of training. If the current round is the first round of training, for each computing node in each training party, use the node index identifier of the computing node as the data index identifier used by the computing node in the current training round.

6. The method according to one of claims 1 to 5, wherein After determining the data used by each computing node in each training round in the current training iteration, the method further includes: In the current round of longitudinal federated training, the data used by this computing node in the current round of training is used in the longitudinal federated training.

7. An apparatus for determining training data, comprising: The acquisition unit is configured to acquire the training dataset; The distribution unit is configured to, for each of at least two trainers participating in longitudinal federated training, cyclically distribute data in the training dataset to each computing node in that trainer based on the data identifier. The first determining unit is configured to, for each training side, determine the amount of data acquired by each computing node in each training round in each training round, based on a preset amount of data and the total number of computing nodes contained in the training side. The second determining unit is configured to, if the current round is not the first round of training, determine the data index identifier used by the computing node in the current round of training for each computing node in each training party, based on the preset data volume, the total number of computing nodes included in the training party, and the data index identifier used by the computing node in the previous round of training. The third determining unit is configured to determine the training data used by the computing node in the current training round based on the amount of data acquired by the computing node in each training round and the data index identifier used by the computing node in the current training round.

8. The apparatus of claim 7, wherein, The distribution unit includes: The first determining module is configured to determine each computing node of each of the at least two trainers participating in the longitudinal federated training. The distribution module is configured to distribute each piece of data in the training dataset to each computing node in a cyclical manner based on the identifier of the data.

9. The apparatus according to claim 7, wherein, The first determining unit includes: The second determining module is configured to determine the amount of data acquired by each computing node in the training process in each round of training based on the quotient between the preset data amount and the total number of computing nodes included in the training process.

10. The apparatus according to claim 7, wherein, The second determining unit includes: The third determining module is configured to, if the current round is not the first round of training, for each computing node in each training party, take the first remainder obtained by taking the sum of the first difference and the total number of computing nodes in the training party, modulo the total number of computing nodes, and determine the first remainder as the data index identifier used by the computing node in the current round of training. The first difference is the difference between the data index identifier used by the computing node in the previous round of training and the second remainder, and the second remainder is the result obtained by taking the remainder of the preset data amount modulo the total number of computing nodes.

11. The apparatus according to claim 7, wherein, The device further includes: The fourth determining unit is configured to determine whether the current training round is the first training round. If the current round is the first training round, for each computing node in each training party, the node index identifier of the computing node is used as the data index identifier used by the computing node in the current training round.

12. The apparatus according to any one of claims 7-11, wherein, The device further includes: The training unit is configured to participate in the longitudinal federated training in the current round using the data used by the computing node in the current round of training.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

Images