Sample sampling method, device, system, electronic device and storage medium

Abstract

This disclosure provides a sample sampling method, apparatus, system, electronic device, and storage medium, relating to artificial intelligence fields such as deep learning. The method may include: during the training of a federated gradient boosting decision tree algorithm model, a first participant uses a gradient-based sampling method to sample samples from a corresponding first sample set, obtaining sampled samples; the first participant sends the sample identifiers of the sampled samples to a second participant, who then uses the samples corresponding to the sample identifiers in a corresponding second sample set as sampled samples. These sampled samples are used for model training. The second sample set corresponds one-to-one with the sample identifiers in the first sample set, and the number of second participants is N-1, where N is a positive integer greater than one, representing the number of participants in the training. Applying the scheme described in this disclosure can reduce computational overhead.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to sample sampling methods, apparatus, systems, electronic devices and storage media in the field of deep learning. Background Technology

[0002] Gradient Boosting Decision Tree (GBDT) is a widely used machine learning algorithm. Like any machine learning algorithm, data is essential. With the strengthening of privacy protection laws and regulations and increasing public awareness of data privacy, the GBDT algorithm faces the problem of data silos—data being fragmented across different companies and departments. Direct data exchange carries the risk of user data privacy breaches.

[0003] To address the aforementioned data silos and privacy breaches, the Federated GBDT algorithm was proposed. This algorithm uses cryptographic methods to collaboratively train multiple parties, allowing them to train the algorithm model without directly sharing plaintext data. However, the training process of the Federated GBDT algorithm is computationally expensive and slow, which limits its widespread application. Summary of the Invention

[0004] This disclosure provides sample collection methods, apparatus, systems, electronic devices, and storage media.

[0005] A sample sampling method, comprising:

[0006] During the training process of the federated gradient boosting decision tree algorithm model, the first participant uses a gradient-based sampling method to sample the samples in the corresponding first sample set to obtain the sampled samples.

[0007] The first participant sends the sample identifier of the sampled sample to the second participant, so that the second participant can use the sample corresponding to the sample identifier in the corresponding second sample set as the sampled sample. The sampled sample is used for model training. The second sample set corresponds one-to-one with the sample identifier in the first sample set. The number of the second participants is N-1, where N is a positive integer greater than one, representing the number of participants in the training.

[0008] A sample sampling method, comprising:

[0009] During the training process of the federated gradient boosting decision tree algorithm model, the second participant obtains the sample identifier from the first participant. The sample identifier is the sample identifier of the sample sampled by the first participant after sampling the samples in the corresponding first sample set using a gradient-based sampling method.

[0010] The second participant takes the samples corresponding to the sample identifier in the corresponding second sample set as the sampled samples. The second sample set corresponds one-to-one with the sample identifiers in the first sample set. The sampled samples are used for model training.

[0011] A sample sampling device includes: a first sampling module and an information transmission module;

[0012] The first sampling module is used to sample samples in the first sample set using a gradient-based sampling method during the training process of the federated gradient boosting decision tree algorithm model, and obtain the sampled samples. The first sample set is the sample set corresponding to the first participant.

[0013] The information sending module is used to send the sample identifier of the sampled sample to the second participant, so that the second participant can use the sample corresponding to the sample identifier in the corresponding second sample set as the sampled sample. The sampled sample is used for model training. The second sample set corresponds one-to-one with the sample identifier in the first sample set. The number of the second participants is N-1, where N is a positive integer greater than one, representing the number of participants in the training.

[0014] A sample sampling device includes: an information acquisition module and a second sampling module;

[0015] The information acquisition module is used to acquire sample identifiers from the first participant during the training process of the federated gradient boosting decision tree algorithm model. The sample identifiers are the sample identifiers of the samples sampled by the first participant after sampling the samples in the corresponding first sample set using a gradient-based sampling method.

[0016] The second sampling module is used to take the samples corresponding to the sample identifier in the second sample set as the sampled samples. The second sample set is the sample set corresponding to the second participant. The second sample set corresponds one-to-one with the sample identifier in the first sample set. The sampled samples are used for model training.

[0017] A sample sampling system includes: the two sample sampling devices as described above.

[0018] An electronic device, comprising:

[0019] At least one processor; and

[0020] A memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described above.

[0022] A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the methods described above.

[0023] A computer program product includes a computer program / instructions that, when executed by a processor, implement the method described above.

[0024] One embodiment disclosed above has the following advantages or beneficial effects: by sampling, the number of samples used for training the algorithm model can be reduced, thereby reducing computational overhead and improving training speed, that is, achieving the goal of accelerating training. Moreover, by adopting a gradient-based sampling method, it can be ensured that the sampled samples are helpful for training improvement, thereby ensuring the accuracy of the model, etc.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0026] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0027] Figure 1 This is a flowchart of the first embodiment of the sample sampling method described in this disclosure;

[0028] Figure 2 This is a flowchart of the second embodiment of the sample sampling method described in this disclosure;

[0029] Figure 3 This is a flowchart of the third embodiment of the sample sampling method described in this disclosure;

[0030] Figure 4 This is a schematic diagram of the composition structure of the first embodiment 400 of the sample sampling device described in this disclosure;

[0031] Figure 5 This is a schematic diagram of the composition structure of the second embodiment 500 of the sample sampling device described in this disclosure;

[0032] Figure 6 This is a schematic diagram of the composition structure of embodiment 600 of the sample sampling system described in this disclosure;

[0033] Figure 7 A schematic block diagram of an electronic device 700 that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0034] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0035] Furthermore, it should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0036] Figure 1 This is a flowchart of the first embodiment of the sample collection method described in this disclosure. Figure 1 As shown, the specific implementation methods are as follows.

[0037] In step 101, during the training of the federated GBDT algorithm model, the first participant uses a gradient-based sampling method to sample the samples in the corresponding first sample set to obtain the sampled samples.

[0038] In step 102, the first participant sends the sample identifier of the sampled sample to the second participant, so that the second participant can use the sample corresponding to the sample identifier in the second sample set as the sampled sample. The sampled sample is used for model training. The sample identifier in the second sample set corresponds one-to-one with the sample identifier in the first sample set. The number of the second participants is N-1, where N is a positive integer greater than one, representing the number of participants in the training.

[0039] As can be seen, by adopting the scheme described in the above method embodiment, the number of samples used for training the algorithm model can be reduced through sample sampling, thereby reducing computational overhead and improving training speed. Moreover, by adopting a gradient-based sampling method, it can be ensured that the sampled samples are helpful for training improvement, thereby ensuring the accuracy of the model.

[0040] In practical applications, the number of participants in the training can be determined according to actual needs, but it must be at least two, meaning that the number of second participants must be at least one.

[0041] The scheme described in this disclosure is particularly applicable to situations where only one participant in the training has labeled samples. Accordingly, the participant can determine the gradient corresponding to each sample (i.e., the training sample) based on the label. There are no restrictions on how the gradient corresponding to each sample is determined. In addition, the gradient corresponding to each sample can be re-determined each time samples are sampled.

[0042] For ease of description, the participants with labeled samples are referred to as the first participants, and the other participants are referred to as the second participants. The first and second participants each correspond to their own sample sets, namely the first sample set and the second sample set. The samples in the first and second sample sets are aligned, meaning that the sample identifiers (ids) in the two sample sets correspond one-to-one.

[0043] During the training of the federated GBDT algorithm model, sample sampling can be performed for each training round, and the sampled samples can then be used for model training. Alternatively, sample sampling can be performed in some rounds, while all samples are used for model training in other rounds. Regardless of the method used, when sample sampling is required, it can be performed according to... Figure 1 The process is performed as shown.

[0044] First, the first participant can use a gradient-based sampling method to sample the samples in the first sample set, thereby obtaining the sampled samples.

[0045] In one embodiment of this disclosure, for any sample in the first sample set, the following processing can be performed: the sample is taken as the sample to be processed, the sampling probability of the sample to be processed is determined according to the gradient corresponding to the sample to be processed, the sampling result of the sample to be processed is determined according to the sampling probability, and if the sampling result is a predetermined value, the sample to be processed is taken as the sampled sample.

[0046] In one embodiment of this disclosure, the sampling probability of the sample to be processed can be determined by either method one or method two.

[0047] 1) Method 1

[0048] In this method, the gradient corresponding to the sample to be processed can be normalized, and the sampling probability of the sample to be processed can be determined based on the normalized gradient, the predetermined sampling rate, and the number of samples in the first sample set.

[0049] In one embodiment of this disclosure, the gradient corresponding to the sample to be processed can be normalized according to the principle that the sum of the normalized gradients of each sample in the first sample set is 1.

[0050] For example, it could be:

[0051] Among them, g i Represents the gradient before normalization, π i represents the normalized gradient, and n represents the number of samples in the first sample set.

[0052] The above are merely illustrative examples and are not intended to limit the technical solutions disclosed herein. In practical applications, other normalization methods may also be adopted, as long as they meet the requirements. It's very flexible and convenient.

[0053] Then, the sampling probability of the sample to be processed can be determined based on the normalized gradient, the predetermined sampling rate, and the number of samples in the first sample set.

[0054] For example, the product of the normalized gradient, the predetermined sampling rate, and the number of samples in the first sample set can be calculated, and the result can be used as the sampling probability of the sample to be processed.

[0055] That is, p i =r*n*π i (2)

[0056] Where, p i represents the sampling probability, that is, the probability that a sample is sampled. r represents the sampling rate, which can be preset and the sampling rate can be different for different rounds.

[0057] 2) Method Two

[0058] In this method, the gradient corresponding to the sample to be processed can be normalized, and differential privacy noise can be added to the normalized gradient. Then, the sampling probability of the sample to be processed can be determined based on the gradient after adding noise, the predetermined sampling rate, and the number of samples in the first sample set.

[0059] Gradient-based sampling may leak the privacy information of the samples. In this method, differential privacy mechanism can be combined to protect the privacy information of the first participant.

[0060] In one embodiment of this disclosure, the gradient corresponding to the sample to be processed can be normalized according to the principle that the sum of the normalized gradients of each sample in the first sample set is 1. For example, the gradient corresponding to the sample to be processed can be normalized according to the method shown in formula (1) above.

[0061] Subsequently, differential privacy noise can be added to the normalized gradient. In one embodiment of this disclosure, the differential privacy noise may include Laplace noise or Gaussian noise. Of course, other noise may be used if needed. This disclosure does not limit the specific type of noise used for the differential privacy noise.

[0062] Using Laplace noise as an example, differential privacy noise can be added to the normalized gradient in the following way: π i =π i +Laplace(1 / ε); (3)

[0063] Where ε represents the privacy budget, which can be preset, and π i ' represents the gradient after adding differential privacy noise.

[0064] Then, the sampling probability of the sample to be processed can be determined based on the gradient after adding noise, the predetermined sampling rate, and the number of samples in the first sample set.

[0065] For example, the product of the gradient after adding noise, the predetermined sampling rate, and the number of samples in the first sample set can be calculated. The result can be used as the sampling probability of the sample to be processed. The sampling probability can be calculated according to formula (2), except that π is used instead of the product of the gradient after adding noise, the sample rate, and the number of samples in the first sample set. i Replace with π i '.

[0066] Generally speaking, the sampling probability needs to be within the range of [0, 1]. However, due to the addition of noise, the obtained sampling probability may not be within the range of [0, 1]. Accordingly, in one embodiment of this disclosure, if the sampling probability is less than 0, 0 can be used as the sampling probability; if the sampling probability is greater than 1, 1 can be used as the sampling probability; if the sampling probability is other values, the original value can be kept unchanged.

[0067] This allows the sampling probability to be clipped to the range [0, 1], thus facilitating subsequent calculations.

[0068] The specific sampling method, whether method one or method two, determines the sampling probability of the sample to be processed, which is very flexible and convenient depending on the actual needs. Preferably, method two can be used, which adds noise to the gradient to meet the differential privacy requirements. The noise level can be customized and controlled through the privacy budget ε.

[0069] After determining the sampling probability of the sample to be processed, the sampling result of the sample to be processed can be determined based on the sampling probability.

[0070] In one embodiment of this disclosure, the sampling probability can be used as a parameter to determine the sampling result according to the Bernoulli distribution. For example, the sampling probability p can be... i The parameter si is used to sample from the Bernoulli distribution, and si is the sampling result.

[0071] Furthermore, if the sampling result is a predetermined value, the sample to be processed can be used as the sampled sample. In one embodiment of this disclosure, the sampling result may include 1 and 0. Accordingly, if the sampling result is 1, the sample to be processed can be used as the sampled sample; conversely, if the sampling result is 0, the sample to be processed will not be used as the sampled sample.

[0072] Through the above processing, it is possible to efficiently and accurately determine whether the sample to be processed is a sampled sample, thereby realizing the sample sampling of the first participant, and the number of samples sampled is approximately n*r.

[0073] The first participant can also synchronize the sampling results to the second participant. That is, the first participant can send the sample identifier of the sampled sample to the second participant, so that the second participant can take the sample corresponding to the sample identifier in the corresponding second sample set as the sampled sample.

[0074] As can be seen, the scheme described in this disclosure adopts a joint sampling method, which ensures the consistency and privacy of the samples collected by both parties.

[0075] Figure 2 This is a flowchart of a second embodiment of the sample collection method described in this disclosure. Figure 2 As shown, the specific implementation methods are as follows.

[0076] In step 201, during the training of the federated GBDT algorithm model, the second participant obtains the sample identifier from the first participant. The sample identifier is the sample identifier of the sample obtained by the first participant after sampling the samples in the corresponding first sample set using a gradient-based sampling method.

[0077] In step 202, the second participant takes the sample corresponding to the sample identifier in the corresponding second sample set as the sampled sample. The second sample set corresponds one-to-one with the sample identifier in the first sample set, and the sampled sample is used for model training.

[0078] For example, if both the first and second sample sets contain 1000 samples (in reality, this may be much larger, but it is just an example), and the first participant samples 300 samples from the first sample set, then the sample identifiers of these 300 samples can be sent to the second participant. Correspondingly, the second participant can use the samples corresponding to these 300 sample identifiers in the second sample set as the sampled samples.

[0079] By adopting the scheme described in the above method embodiment, the number of samples used for training the algorithm model can be reduced through sample sampling, thereby reducing computational overhead and improving training speed. Moreover, by adopting a gradient-based sampling method, it can be ensured that the sampled samples are helpful for training improvement, thereby ensuring the accuracy of the model.

[0080] Figure 3 This is a flowchart of a third embodiment of the sample sampling method described in this disclosure. The method can be applied to the training process of the Federated GBDT algorithm model. Figure 3 As shown, the specific implementation methods may include the following.

[0081] In step 301, the first participant processes each sample in the first sample set according to the methods shown in steps 302-307.

[0082] In step 302, the sample is used as the sample to be processed, and the gradient corresponding to the sample to be processed is normalized.

[0083] For example, the gradients of the samples to be processed can be normalized according to the principle that the sum of the normalized gradients of each sample in the first sample set is 1.

[0084] In step 303, differential privacy noise is added to the normalized gradient.

[0085] The differential privacy noise may include Laplace noise or Gaussian noise.

[0086] In step 304, the sampling probability of the sample to be processed is determined based on the gradient after adding noise, the predetermined sampling rate, and the number of samples in the first sample set.

[0087] For example, the product of the gradient after adding noise, the predetermined sampling rate, and the number of samples in the first sample set can be calculated, and the result can be used as the sampling probability of the sample to be processed.

[0088] In step 305, the sampling probability is clipped to the range [0, 1].

[0089] For example, if the sampling probability is less than 0, then 0 can be used as the sampling probability; if the sampling probability is greater than 1, then 1 can be used as the sampling probability.

[0090] In step 306, the sampling probability is used as a parameter to determine the sampling result of the sample to be processed according to the Bernoulli distribution.

[0091] The sampling results can include 1 and 0.

[0092] In step 307, if the sampling result is 1, the sample to be processed is used as the sampled sample; if the sampling result is 0, the sample to be processed is not used as the sampled sample.

[0093] In step 308, the sample identifier of the sampled sample is sent to the second participant.

[0094] In step 309, the second participant takes the sample corresponding to the sample identifier in the corresponding second sample set as the sampled sample.

[0095] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this disclosure. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure. Furthermore, for parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0096] In summary, the scheme described in the embodiments of this disclosure adopts a gradient-based sampling method, which can greatly reduce computational overhead and improve training speed while ensuring model accuracy. At the same time, it incorporates a differential privacy mechanism, which can effectively protect the privacy information of the labeler.

[0097] The above is an introduction to the method embodiments. The following describes the solution described in this disclosure further through device embodiments.

[0098] Figure 4 This is a schematic diagram of the composition structure of the first embodiment 400 of the sample sampling device described in this disclosure. Figure 4 As shown, it includes: a first sampling module 401 and an information sending module 402.

[0099] The first sampling module 401 is used to sample samples in the first sample set using a gradient-based sampling method during the training of the federated GBDT algorithm model, and obtain the sampled samples. The first sample set is the sample set corresponding to the first participant.

[0100] The information sending module 402 is used to send the sample identifier of the sampled sample to the second participant, so that the second participant can take the sample corresponding to the sample identifier in the corresponding second sample set as the sampled sample, and the sampled sample is used for model training. The second sample set corresponds one-to-one with the sample identifier in the first sample set, and the number of the second participants is N-1, where N is a positive integer greater than one, representing the number of participants in the training.

[0101] By adopting the scheme described in the above-described device embodiment, the number of samples used for training the algorithm model can be reduced through sample sampling, thereby reducing computational overhead and improving training speed. Moreover, by adopting a gradient-based sampling method, it can be ensured that the sampled samples are helpful for training improvement, thereby ensuring the accuracy of the model.

[0102] In one embodiment of this disclosure, the first sampling module 401 may perform the following processing on any sample in the first sample set: take the sample as a sample to be processed, determine the sampling probability of the sample to be processed according to the gradient corresponding to the sample to be processed; determine the sampling result of the sample to be processed according to the sampling probability; if the sampling result is a predetermined value, then take the sample to be processed as the sampled sample.

[0103] In one embodiment of this disclosure, the sampling probability of the sample to be processed can be determined by either method one or method two.

[0104] Method 1:

[0105] The first sampling module 401 can normalize the gradient corresponding to the sample to be processed, and determine the sampling probability of the sample to be processed based on the normalized gradient, the predetermined sampling rate and the number of samples in the first sample set.

[0106] In one embodiment of this disclosure, the first sampling module 401 can normalize the gradient corresponding to the sample to be processed according to the principle that the sum of the normalized gradients of each sample in the first sample set is 1.

[0107] In addition, the first sampling module 401 can calculate the product of the normalized gradient, the predetermined sampling rate, and the number of samples in the first sample set, and use the calculation result as the sampling probability of the sample to be processed.

[0108] Method 2:

[0109] The first sampling module 401 can normalize the gradient corresponding to the sample to be processed, and add differential privacy noise to the normalized gradient, thereby determining the sampling probability based on the gradient after adding noise, the predetermined sampling rate, and the number of samples in the first sample set.

[0110] In one embodiment of this disclosure, the first sampling module 401 can normalize the gradient corresponding to the sample to be processed according to the principle that the sum of the normalized gradients of each sample in the first sample set is 1.

[0111] Subsequently, the first sampling module 401 may add differential privacy noise to the normalized gradient. In one embodiment of this disclosure, the differential privacy noise may include Laplace noise or Gaussian noise.

[0112] Furthermore, the first sampling module 401 can calculate the product of the gradient after adding noise, the predetermined sampling rate, and the number of samples in the first sample set, and use the calculation result as the sampling probability of the sample to be processed.

[0113] Generally speaking, the sampling probability needs to be within the range of [0, 1]. However, due to the addition of noise, the obtained sampling probability may not be within the range of [0, 1]. Accordingly, in one embodiment of this disclosure, if the sampling probability is less than 0, 0 can be used as the sampling probability; if the sampling probability is greater than 1, 1 can be used as the sampling probability; if the sampling probability is other values, the original value can be kept unchanged.

[0114] In one embodiment of this disclosure, the first sampling module 401 can use the sampling probability as a parameter to determine the sampling result according to the Bernoulli distribution. For example, the sampling probability p can be used as a parameter to determine the sampling result according to the Bernoulli distribution. i The parameter si is used to sample from the Bernoulli distribution, and si is the sampling result.

[0115] Furthermore, if the sampling result is a predetermined value, the first sampling module 401 can use the sample to be processed as the sampled sample. In one embodiment of this disclosure, the sampling result may include 1 and 0. Accordingly, if the sampling result is 1, the sample to be processed can be used as the sampled sample; conversely, if the sampling result is 0, the sample to be processed will not be used as the sampled sample.

[0116] The information sending module 402 can synchronize the sampling results to the second participant, that is, send the sample identifier of the sampled sample to the second participant, so that the second participant can take the sample corresponding to the sample identifier in the corresponding second sample set as the sampled sample.

[0117] Figure 5 This is a schematic diagram of the composition structure of the second embodiment 500 of the sample sampling device described in this disclosure. Figure 5 As shown, it includes: an information acquisition module 501 and a second sampling module 502.

[0118] The information acquisition module 501 is used to acquire sample identifiers from the first participant during the training process of the federated GBDT algorithm model. The sample identifiers are the sample identifiers of the samples sampled by the first participant after sampling the samples in the corresponding first sample set using a gradient-based sampling method.

[0119] The second sampling module 502 is used to take the samples corresponding to the sample identifiers in the second sample set as the sampled samples. The second sample set is the sample set corresponding to the second participant. The sample identifiers in the second sample set correspond one-to-one with the sample identifiers in the first sample set. The sampled samples are used for model training.

[0120] By adopting the scheme described in the above-described device embodiment, the number of samples used for training the algorithm model can be reduced through sample sampling, thereby reducing computational overhead and improving training speed. Moreover, by adopting a gradient-based sampling method, it can be ensured that the sampled samples are helpful for training improvement, thereby ensuring the accuracy of the model.

[0121] Figure 6 This is a schematic diagram of the composition structure of embodiment 600 of the sample collection system described in this disclosure. Figure 6 As shown, it includes: a first sample sampling device 601 and a second sample sampling device 602.

[0122] The first sample sampling device 601 can be Figure 4 The sample sampling device in the illustrated embodiment, the second sample sampling device 602 may be: Figure 5 The sample sampling device in the illustrated embodiment.

[0123] For the specific workflow of the above device and system embodiments, please refer to the relevant descriptions in the foregoing method embodiments, which will not be repeated here.

[0124] In summary, the scheme described in the embodiments of the device and system disclosed herein adopts a gradient-based sampling method, which can greatly reduce computational overhead and improve training speed while ensuring model accuracy. At the same time, it integrates a differential privacy mechanism, which can effectively protect the privacy information of the labeler.

[0125] The solutions described in this disclosure can be applied to the field of artificial intelligence, particularly deep learning. Artificial intelligence is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It involves both hardware and software technologies. Artificial intelligence hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0126] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0127] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0128] Figure 7A schematic block diagram of an electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0129] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0130] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0131] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as those described in this disclosure. For example, in some embodiments, the methods described in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the methods described in this disclosure can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the methods described in this disclosure by any other suitable means (e.g., by means of firmware).

[0132] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0133] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0134] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0135] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0136] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0137] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0138] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0139] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A sample sampling method, comprising: During the training process of the federated gradient boosting decision tree algorithm model, the first participant uses a gradient-based sampling method to sample samples in the corresponding first sample set to obtain the sampled samples. This includes: for any sample in the first sample set, performing the following processing: taking the sample as a sample to be processed, normalizing the gradient corresponding to the sample to be processed, adding differential privacy noise to the normalized gradient, and calculating the product of the gradient after adding noise, a predetermined sampling rate, and the number of samples in the first sample set. The product is determined as the sampling probability of the sample to be processed, and the sampling result of the sample to be processed is determined according to the sampling probability. If the sampling result is a predetermined value, then the sample to be processed is taken as the sampled sample. The first participant sends the sample identifier of the sampled sample to the second participant, so that the second participant can use the sample corresponding to the sample identifier in the corresponding second sample set as the sampled sample. The sampled sample is used for model training. The second sample set corresponds one-to-one with the sample identifier in the first sample set. The number of the second participants is N-1, where N is a positive integer greater than one, representing the number of participants in the training.

2. The method according to claim 1, further comprising: If the sampling probability is less than 0, then 0 is taken as the sampling probability; if the sampling probability is greater than 1, then 1 is taken as the sampling probability.

3. The method according to claim 1, wherein, The differential privacy noise includes Laplace noise or Gaussian noise.

4. The method according to any one of claims 1 to 3, wherein, The normalization process for the gradient corresponding to the sample to be processed includes: The gradients of the samples to be processed are normalized according to the principle that the sum of the normalized gradients of each sample in the first sample set is 1.

5. The method according to any one of claims 1 to 3, wherein, The step of determining the sampling result of the sample to be processed based on the sampling probability includes: The sampling probability is used as a parameter to determine the sampling result according to the Bernoulli distribution.

6. The method according to any one of claims 1 to 3, wherein, The sampling results include: 1 and 0; The step of taking the sample to be processed as the sampled sample if the sampling result is a predetermined value includes: taking the sample to be processed as the sampled sample if the sampling result is 1.

7. A sample sampling method, comprising: During the training process of the federated gradient boosting decision tree algorithm model, the second participant obtains the sample identifier from the first participant. The sample identifier is the sample identifier of the sample sampled by the first participant after sampling the samples in the corresponding first sample set using a gradient-based sampling method. The sample identifier of the sampled sample includes: for any sample in the first sample set, taking the sample as the sample to be processed and determining the sampling probability of the sample to be processed based on the gradient corresponding to the sample to be processed and determining the sampling result of the sample to be processed based on the sampling probability, and then determining the sample identifier of the sample to be processed when the sampling result is a predetermined value. The sampling probability is the product of the gradient after adding noise, the predetermined sampling rate, and the number of samples in the first sample set after normalizing the gradient corresponding to the sample to be processed and adding differential privacy noise to the normalized gradient. The second participant takes the samples corresponding to the sample identifier in the corresponding second sample set as the sampled samples. The second sample set corresponds one-to-one with the sample identifiers in the first sample set. The sampled samples are used for model training.

8. A sample collection device, comprising: The first sampling module and the information sending module; The first sampling module is used to sample samples in the first sample set using a gradient-based sampling method during the training of the federated gradient boosting decision tree algorithm model, and obtain the sampled samples. The sampling module includes the following processing for any sample in the first sample set: taking the sample as a sample to be processed, normalizing the gradient corresponding to the sample to be processed, adding differential privacy noise to the normalized gradient, and calculating the product of the gradient after adding noise, a predetermined sampling rate, and the number of samples in the first sample set. The product is determined as the sampling probability of the sample to be processed, and the sampling result of the sample to be processed is determined according to the sampling probability. If the sampling result is a predetermined value, the sample to be processed is taken as the sampled sample. The first sample set is the sample set corresponding to the first participant. The information sending module is used to send the sample identifier of the sampled sample to the second participant, so that the second participant can use the sample corresponding to the sample identifier in the corresponding second sample set as the sampled sample. The sampled sample is used for model training. The second sample set corresponds one-to-one with the sample identifier in the first sample set. The number of the second participants is N-1, where N is a positive integer greater than one, representing the number of participants in the training.

9. The apparatus according to claim 8, wherein, The first sampling module is further configured to, if the sampling probability is less than 0, use 0 as the sampling probability, and if the sampling probability is greater than 1, use 1 as the sampling probability.

10. The apparatus according to claim 8, wherein, The differential privacy noise includes Laplace noise or Gaussian noise.

11. The apparatus according to any one of claims 8 to 10, wherein, The first sampling module normalizes the gradient corresponding to the sample to be processed according to the principle that the sum of the normalized gradients of each sample in the first sample set is 1.

12. The apparatus according to any one of claims 8 to 10, wherein, The first sampling module uses the sampling probability as a parameter to determine the sampling result based on the Bernoulli distribution.

13. The apparatus according to any one of claims 8 to 10, wherein, The sampling results include: 1 and 0; If the first sampling module determines that the sampling result is 1, then the sample to be processed is taken as the sampled sample.

14. A sample collection device, comprising: Information acquisition module and second sampling module; The information acquisition module is used to acquire sample identifiers from the first participant during the training process of the federated gradient boosting decision tree algorithm model. The sample identifiers are the sample identifiers of the samples sampled by the first participant after sampling the samples in the corresponding first sample set using a gradient-based sampling method. The sample identifier of the sampled sample includes: for any sample in the first sample set, taking the sample as the sample to be processed and determining the sampling probability of the sample to be processed based on the gradient corresponding to the sample to be processed and determining the sampling result of the sample to be processed based on the sampling probability, and then determining the sample identifier of the sample to be processed when the sampling result is a predetermined value. The sampling probability is the product of the gradient after adding noise, the predetermined sampling rate, and the number of samples in the first sample set after normalizing the gradient corresponding to the sample to be processed and adding differential privacy noise to the normalized gradient. The second sampling module is used to take the samples corresponding to the sample identifiers in the second sample set as the sampled samples. The second sample set is the sample set corresponding to the second participant. The second sample set corresponds one-to-one with the sample identifiers in the first sample set. The sampled samples are used for model training.

15. A sample collection system, comprising: The sample sampling device according to any one of claims 8-13, and the sample sampling device according to claim 14.

16. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

17. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

18. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the method of any one of claims 1-7.

Images