Hybrid architecture system and task scheduling method based on historical task effects

Abstract

The present application discloses a hybrid architecture system and a task scheduling method based on historical task effects, the method comprising: when detecting that the first participant initiates a task to be processed, receiving the associated task information and the information of the first data set sent by the first participant; receiving the associated second data set information sent by the second participant after detecting the task to be processed; determining the task type of the task to be processed and returning it to the first participant and the second participant according to the task information, the information of the first data set, the information of the second data set and the preset task type recommendation model, the task type includes a decentralized type and a centralized type; by adopting a centralized scheduling mode / a decentralized scheduling mode for scheduling, cooperating with the first participant and the second participant to perform privacy calculation on the task to be processed according to the target scheduling mode corresponding to the task type, and obtaining the first processing result. In the present application, the best calculation method is selected to avoid waste.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a hybrid architecture system and a task scheduling method based on historical task effects. Background Art

[0002] With the development of artificial intelligence technology, the application of privacy computing (based on federated learning) is becoming more and more extensive. Privacy computing refers to a technology and system for joint computing by two or more participants (the participants in privacy computing can be different departments of the same organization or different organizations, realizing cross-domain cooperation of multi-source data while protecting data security). The participants conduct joint machine learning and joint analysis on their data through collaboration without leaking their respective data.

[0003] During the privacy computing process, different computing methods can be selected. In the prior art, users manually select a specific computing method. However, the manually selected computing method is often not the most optimal, resulting in a waste of privacy computing resources. Summary of the invention

[0004] The main purpose of this application is to provide a hybrid architecture system and a task scheduling method based on historical task results, aiming to solve the technical problem of manually selecting calculation methods in the existing privacy computing process, which easily leads to waste of privacy computing resources.

[0005] To achieve the above objectives, the present application also provides a hybrid architecture system, the hybrid architecture system comprising:

[0006] A first participant, the first participant is used to: initiate a task to be processed, and send task information of the task to be processed and information of a first data set associated with the task to be processed to a third party; and also to send the first data set to the third party when the task to be processed is of a centralized type, or to coordinate with the third party and other participants to perform distributed computing through local task scheduling when the task to be processed is of a distributed type;

[0007] a second participant, the second participant being used to: after detecting that the first participant has initiated a task to be processed, send information of a second data set associated with the task to be processed to a third party; and also being used to send the second data set to the third party when the task to be processed is of a centralized type, or to coordinate with the third party and other participants to perform distributed computing through local task scheduling when the task to be processed is of a distributed type;

[0008] A third party, the third party is used to: determine the task type of the task to be processed based on the task information, the first data set information and the second data set information, wherein the task type includes a decentralized type and a centralized type; and is also used to perform centralized computing in combination with the first data set and the second data set when the task to be processed is of a centralized type, or to perform decentralized computing through local computing in coordination with other participants when the task to be processed is of a decentralized type.

[0009] Optionally, the first participant includes a first scheduling module, a first computing module and a first communication module. The first scheduling module is used to send the task information of the task to be processed and the information of the first data set associated with the task to be processed to a third party through the first communication module after initiating the task to be processed, and is also used to schedule the first computing module to perform local calculations through local tasks to cooperate with the third party and other participants to perform distributed calculations when the task to be processed is a distributed type, or to send the first data set to the third party through the first communication module when the task to be processed is a centralized type.

[0010] Optionally, the second participant includes a second scheduling module, a second computing module and a second communication module, and the second scheduling module is used to: after detecting that the first participant initiates a task to be processed, send information of the second data set associated with the task to be processed to a third party through the second communication module, and also to, when the task to be processed is a distributed type, schedule the second computing module to perform local calculations through local tasks to cooperate with the third party and other participants to perform distributed calculations, or when the task to be processed is a centralized type, send the second data set to the third party through the second communication module.

[0011] Optionally, the third party includes a third scheduling module, a third computing module, a third communication module and a task type recommendation module, the third communication module is used to communicate with the first participant and / or the second participant; the task type recommendation module is used to: determine the task type of the task to be processed based on the task information, the information of the first data set and the information of the second data set; the third scheduling module is used to: when the task to be processed is of a centralized type, schedule the third computing module to perform centralized calculations in combination with the first data set and the second data set, or when the task to be processed is of a distributed type, schedule the third computing module to perform local calculations through local tasks to cooperate with other participants to perform distributed calculations.

[0012] The present application also provides a task scheduling method, which is applied to a third party. The task scheduling method based on historical task effects includes:

[0013] When detecting that a first participant participating in federated learning initiates a task to be processed, receiving task information of the task to be processed and information of a first data set associated with the task to be processed sent by the first participant;

[0014] receiving information of a second data set associated with the task to be processed and sent by the second participant after detecting the task to be processed;

[0015] Determine the task type of the task to be processed according to the task information, the information of the first data set, the information of the second data set, and a preset task type recommendation model, wherein the task type includes a decentralized type and a centralized type;

[0016] The task type is returned to the first participant and the second participant, and the first participant and the second participant cooperate to perform privacy calculation on the task to be processed according to the target scheduling mode corresponding to the task type by adopting a centralized scheduling mode to schedule the centralized task / adopting a decentralized scheduling mode to schedule the decentralized type task, and obtain a first processing result.

[0017] Optionally, the step of determining the task type of the task to be processed according to the task information, the information of the first data set, the information of the second data set, and a preset task type recommendation model includes:

[0018] Inputting the task information, the information of the first data set and the information of the second data set into a preset task type recommendation model;

[0019] The preset task type recommendation model is obtained by iterative training based on a preset training data set with task effect score labels;

[0020] Based on the preset task type recommendation model, prediction processing is performed on the task information, the information of the first data set, and the information of the second data set to obtain a prediction effect score;

[0021] Based on the prediction effect score, the task type of the task to be processed is determined.

[0022] Optionally, before the step of performing prediction processing on the task information, the information of the first data set, and the information of the second data set based on the preset task type recommendation model to obtain a prediction effect score, the method includes:

[0023] Obtaining a preset training data set with a task effect score label and a model to be trained, wherein the preset training data set has at least characteristic factors of task type, algorithm type and data attribute;

[0024] Determining a training matrix of the preset training data set based on the characteristic factors of the task type, algorithm type and data attributes;

[0025] Inputting the training matrix into the model to be trained, and performing prediction processing on the training matrix based on the model to be trained to obtain a prediction result;

[0026] Comparing the prediction result with the task effect score label and obtaining a comparison result;

[0027] Based on the comparison result, determining whether the model to be trained has completed training;

[0028] If the training is not completed, the parameters of the model to be trained are adjusted, and the training matrix is ​​returned to be input into the model to be trained, and the training matrix is ​​predicted based on the model to be trained to obtain the prediction result, until a preset task type recommendation model that meets the preset training completion conditions is obtained.

[0029] Optionally, the task scheduling method based on historical task effects further includes:

[0030] Determine the number of tasks that have been scheduled in the historical period;

[0031] If the number of the scheduled tasks is less than a preset number, a default task type is read from the configuration information of the system, wherein the preset number is obtained by configuration, and the default task type includes a decentralized type and a centralized type;

[0032] Determine a target scheduling mode corresponding to the default task type to process the to-be-processed task based on the target scheduling mode and obtain a second processing result.

[0033] Optionally, the method of scheduling the centralized task by adopting a centralized scheduling mode, and cooperating with the first participant and the second participant to perform privacy calculation on the task to be processed according to the target scheduling mode corresponding to the task type, and obtaining the first processing result includes:

[0034] Determine whether a first data set of the first participant and a second data set of the second participant are stored locally;

[0035] If not, scheduling the first participant to encrypt and upload the first data set, and scheduling other second participants to encrypt and upload the second data set, and saving the first data set and the second data set;

[0036] The first data set and the second data set are retrieved locally, and centralized computing is performed on the task to be processed to obtain a first processing result.

[0037] Optionally, the method of scheduling the decentralized type tasks by adopting a decentralized scheduling mode, and cooperating with the first participant and the second participant to perform privacy calculation on the to-be-processed task according to the target scheduling mode corresponding to the task type, and obtaining the first processing result includes:

[0038] If the decentralized type of the task to be processed is a horizontal federated decentralized type, receiving first intermediate data encrypted and sent by each participant;

[0039] The first intermediate data is generated by each participant based on the corresponding first model gradient, and the first model gradient is obtained by each participant performing privacy calculation based on the corresponding local model;

[0040] aggregating the first intermediate data to obtain aggregated intermediate data;

[0041] Sending the aggregated intermediate data to each participant, so that each participant updates their own model parameters based on the aggregated intermediate data, obtains their own updated local models, and performs privacy calculations based on their own updated local models to continue to generate new first intermediate data;

[0042] Iterate and return to the step of receiving the first intermediate data sent by each participant until the first preset completion condition is met, and take the data result corresponding to the first preset completion condition as the first processing result.

[0043] Optionally, the method of scheduling the decentralized type tasks by adopting a decentralized scheduling mode, and cooperating with the first participant and the second participant to perform privacy calculation on the to-be-processed task according to the target scheduling mode corresponding to the task type, and obtaining the first processing result includes:

[0044] If the decentralized type of the task to be processed is a vertical federation type, receiving the intermediate results sent by each participant after encrypting and masking the second intermediate data;

[0045] The second intermediate data is generated by each participant based on the corresponding encrypted second model gradient exchange, and the second model gradient is calculated by each participant based on the corresponding local model;

[0046] Decrypting the intermediate result, and sending the decrypted intermediate result to the corresponding participant, so that each participant can update their own model parameters based on the intermediate result, obtain their own updated local models, and continue to generate new intermediate results based on their own updated local models;

[0047] Iterate and return to the step of receiving the intermediate results sent by each participant after encrypting and masking the second intermediate data, until the second preset completion condition is met, and the data result corresponding to the second preset completion condition is used as the first processing result.

[0048] The present application also provides a task scheduling method, which is applied to any first participant in a participant subsystem that initiates a task to be processed, wherein the participant subsystem also includes at least one second participant associated with the task to be processed, and the task scheduling method based on historical task effects includes:

[0049] When initiating a task to be processed, determining task information of the task to be processed and information of a first data set associated with the task to be processed, and sending the task information and information of the first data set to a third party;

[0050] receiving the task type of the task to be processed sent by the third party, and when the task to be processed is of a centralized type, sending the first data set to the third party, so that the third party can perform centralized computing on the first data set and the second data set uploaded by the second participant after receiving the task type, to obtain a first processing result; or, when the task to be processed is of a decentralized type, performing decentralized computing in coordination with the third party and other participants through local task scheduling, to obtain the first processing result;

[0051] Among them, the task type is determined by the third party based on the task information, the information of the first data set, the information of the second data set and a preset task type recommendation model, the task type includes a decentralized type and a centralized type, the information of the second data set is sent to the third party by the second participant after detecting the task to be processed, and the second data set is associated with the task to be processed.

[0052] Optionally, the method further comprises:

[0053] After the processing of the task to be processed is completed, the task processing process of the task to be processed is scored according to the processing effect, and the task information, task type and the score of the task to be processed are sent to the task type recommendation module of the third party accordingly, so that the third party stores the task information, the task type, the score of the task to be processed, and the first data set information and the second data set information associated with the task to be processed accordingly based on the task information.

[0054] The present application also provides a task scheduling method, which is applied to any second participant in a participant subsystem that is passively associated with a task to be processed, wherein the participant subsystem also includes a first participant that initiates the task to be processed. The task scheduling method based on historical task effects includes:

[0055] When a task to be processed is monitored, determining information of a second data set associated with the task to be processed, and sending the information of the second data set to a third party;

[0056] receiving the task type of the task to be processed sent by the third party, and when the task to be processed is of a centralized type, sending the second data set to the third party, so that the third party can perform centralized computing on the basis of the second data set and the first data set uploaded by the first participant after receiving the task type, to obtain a first processing result; or, when the task to be processed is of a decentralized type, performing decentralized computing in cooperation with the third party and other participants through local task scheduling, to obtain the first processing result;

[0057] Among them, the task type is determined by the third party based on the task information, the information of the first data set, the information of the second data set and a preset task type recommendation model, the task type includes a decentralized type and a centralized type, the information of the first data set is sent to the third party by the first participant after initiating the task to be processed, and the first data set is associated with the task to be processed.

[0058] Optionally, the task scheduling method based on historical task effects further includes:

[0059] When in an idle state, the local first data set is uploaded to the third party.

[0060] Optionally, when the task to be processed is of a distributed type, the step of obtaining a first processing result by cooperating with a third party and other participants to perform distributed computing through local task scheduling includes:

[0061] If the decentralized type of the task to be processed is a horizontal federation type, sending first intermediate data to the third party; wherein the first intermediate data is generated by each participant based on a corresponding first model gradient, and the first model gradient is calculated by each participant based on a corresponding local model;

[0062] Receiving aggregated intermediate data sent by a third party, wherein the aggregated intermediate data is obtained by the third party aggregating the first intermediate data;

[0063] Updating model parameters based on the aggregated intermediate data to obtain updated local models, and continuing to generate new first intermediate data based on the respective updated local models;

[0064] Iterate and return to the step of sending the first intermediate data to the third party until a first preset completion condition is met, and use the data result corresponding to the first preset completion condition as the first processing result.

[0065] Optionally, when the task to be processed is of a distributed type, the step of obtaining a first processing result by cooperating with a third party and other participants to perform distributed computing through local task scheduling includes:

[0066] If the decentralized type of the task to be processed is a vertical federation type, encrypting and masking the second intermediate data to obtain an intermediate result, and sending the intermediate result to the third party;

[0067] The second intermediate data is generated by each participant based on the corresponding encrypted second model gradient exchange, and the second model gradient is calculated by each participant based on the corresponding local model;

[0068] After receiving the decrypted intermediate result sent by the third party after decryption processing of the intermediate result, updating the respective model parameters based on the decrypted intermediate result, and obtaining the respective updated local models, and continuing to generate new intermediate results based on the respective updated local models;

[0069] Iterate and return to the step of encrypting the second intermediate data and obtaining the intermediate result after masking, until the second preset completion condition is met, and take the data result corresponding to the second preset completion condition as the first processing result.

[0070] The present application also provides an electronic device, which is a physical node device, and includes: a memory, a processor, and a program of the task scheduling method based on historical task effects stored in the memory and executable on the processor. When the program of the task scheduling method based on historical task effects is executed by the processor, the steps of the task scheduling method based on historical task effects as described above can be implemented.

[0071] The present application also provides a storage medium, on which is stored a program for implementing the above-mentioned task scheduling method. When the program of the task scheduling method based on historical task effects is executed by a processor, the steps of the task scheduling method based on historical task effects as described above are implemented.

[0072] The present application also provides a product, which is a computer program product, including a computer program, which implements the steps of the above-mentioned task scheduling method based on historical task effects when executed by a processor.

[0073] The present application provides a hybrid architecture system and a task scheduling method based on historical task effects. In the present application, when it is detected that a first participant participating in federated learning initiates a task to be processed, task information of the task to be processed and information of a first data set associated with the task to be processed sent by the first participant are received; information of a second data set associated with the task to be processed sent by the second participant after detecting the task to be processed is received; the task type of the task to be processed is determined according to the task information, the information of the first data set, the information of the second data set, and a preset task type recommendation model, wherein the task type includes a decentralized type and a centralized type; the task type is returned to the first participant and the second participant, and by adopting a centralized scheduling mode to schedule centralized tasks / adopting a decentralized scheduling mode to schedule decentralized type tasks, the first participant and the second participant cooperate to perform privacy calculation on the task to be processed according to the target scheduling mode corresponding to the task type, and obtain a first processing result. In the present application, a process for determining the task type corresponding to the task to be processed is added based on task information, information of the first data set, information of the second data set, and a preset task type recommendation model, and then task processing is performed on the task to be processed based on the corresponding scheduling modes of different task types (distributed type or centralized type). It can be understood that the specific calculation method is not selected by humans based on their own assumptions, but the task type corresponding to the task to be processed is clarified through the preset task type recommendation model (and the preset task type recommendation model is determined based on the processing effect or score of the corresponding historical task), and then the calculation method corresponding to the task type is selected, that is, the calculation method with the best effect, to avoid wasting privacy computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0075] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0076] Figure 1 This is a flowchart of the first embodiment of the task scheduling method based on historical task effects of the present application;

[0077] Figure 2 This is a detailed flow chart of step S30 in the first embodiment of the task scheduling method based on historical task effects of this application;

[0078] Figure 3 A schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application;

[0079] Figure 4 This is a schematic diagram of a first scenario of centralized computing of centralized type tasks involved in the task scheduling method based on historical task effects of this application;

[0080] Figure 5 A schematic diagram of a second scenario of performing distributed computing when the distributed type of the task to be processed is a horizontal federated distributed type, according to the task scheduling method based on historical task effects of the present application;

[0081] Figure 6 A schematic diagram of a third scenario of performing distributed computing when the distributed type of the task to be processed is a vertical federated distributed type involved in the task scheduling method based on historical task effects of the present application;

[0082] Figure 7 A schematic diagram of the entire process of the task scheduling method based on historical task effects involved in the task scheduling method based on historical task effects of this application;

[0083] Figure 8 A schematic diagram of a hybrid architecture system of the present application.

[0084] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0085] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0086] Embodiment 1

[0087] The present application embodiment provides a task scheduling method. In the first embodiment of the task scheduling method of the present application, refer to Figure 1 , applied to a third party, the task scheduling method based on historical task effects includes:

[0088] Step S10, when it is detected that a first participant participating in federated learning initiates a task to be processed, receiving task information of the task to be processed and information of a first data set associated with the task to be processed sent by the first participant;

[0089] Step S20, receiving information of a second data set associated with the task to be processed, which is sent by the second participant after detecting the task to be processed;

[0090] Step S30, determining the task type of the task to be processed according to the task information, the information of the first data set, the information of the second data set, and a preset task type recommendation model, wherein the task type includes a decentralized type and a centralized type;

[0091] Step S40, returning the task type to the first participant and the second participant, and cooperating with the first participant and the second participant to perform privacy calculation on the task to be processed according to the target scheduling mode corresponding to the task type by adopting a centralized scheduling mode to schedule the centralized task / adopting a decentralized scheduling mode to schedule the decentralized type task, and obtaining a first processing result.

[0092] In this embodiment, a determination process of the task type corresponding to the task to be processed is added based on the task information, the information of the first data set, the information of the second data set and the preset task type recommendation model, and then the task to be processed is processed based on the scheduling modes corresponding to different task types (distributed type or centralized type). It can be understood that the specific calculation method is not selected by humans as it is, but the task type corresponding to the task to be processed is clarified through the preset task type recommendation model (and the preset task type recommendation model is determined according to the processing effect or score of the corresponding historical task), and then the calculation method corresponding to the task type is selected, that is, the calculation method with the best effect, to avoid wasting privacy computing resources.

[0093] The specific steps are as follows:

[0094] Step S10, when it is detected that a first participant participating in federated learning initiates a task to be processed, receiving task information of the task to be processed and information of a first data set associated with the task to be processed sent by the first participant;

[0095] In this embodiment, it should be noted that the task scheduling method can be applied to a task scheduling device, which belongs to a third party and the third party belongs to a hybrid architecture system.

[0096] It should be noted that the hybrid architecture system includes not only a third party (centralizer, coordinator or aggregator), but also various subsystems (participant subsystems).

[0097] In recent years, the integrated application of data has driven all walks of life towards digitalization, networking and intelligence. In the process of all walks of life moving towards digitalization, networking and intelligence, issues such as data security and personal privacy protection have increasingly received widespread attention from the society. That is, how to achieve data integration under the premise of compliance (ensuring data security and ensuring personal privacy protection) has become a difficult problem that needs to be solved urgently.

[0098] Privacy computing refers to a technology and system for joint computing by two or more parties. The parties collaborate to conduct joint machine learning and joint analysis on their data without disclosing their own data. The participants in privacy computing can be different departments of the same organization or different organizations. Under the privacy computing framework, the data of the participants will not be exported in plain text. While protecting data security, multi-source data cross-domain cooperation is achieved to solve the problem of data protection and integrated application.

[0099] At present, privacy computing mainly includes federated learning computing. Federated learning refers to a method of performing machine learning by uniting different participants. A round of parameter update in federated learning is divided into two steps: (a) each participant only uses its own data to train the machine learning model and sends model parameter updates to a central coordinator; (b) the coordinator integrates the model updates received from different participants (for example, taking the average) and redistributes the integrated model parameter updates to each participant. In federated learning, participants do not need to expose their data to other participants or coordinators. Therefore, federated learning can well protect user privacy and ensure data security.

[0100] Federated learning includes vertical federated learning computing and horizontal federated learning computing. Vertical federated learning is sample-aligned federated learning (Sample-Aligned Federated Learning), which is suitable for situations where the participants' training sample IDs overlap a lot but the data features overlap less. It requires multiple parties to collaborate to complete model training and optimization under a secure and confidential framework.

[0101] Horizontal Federated Learning is suitable for situations where the data features of participants' training samples overlap a lot but there are fewer user IDs. It requires multiple parties to collaborate to complete model training and optimization under a secure and confidential framework.

[0102] In this embodiment, the specific application scenario may be:

[0103] During the privacy computing process, different computing methods can be selected. In the prior art, users manually select a specific computing method. However, the manually selected computing method is often not the most optimal, resulting in a waste of privacy computing resources.

[0104] In this embodiment, a determination process of the task type corresponding to the task to be processed is added based on the task information, the information of the first data set, the information of the second data set and the preset task type recommendation model, and then the task to be processed is processed based on the scheduling modes corresponding to different task types (distributed type or centralized type). It can be understood that the specific calculation method is not selected by humans as it is, but the task type corresponding to the task to be processed is clarified through the preset task type recommendation model (and the preset task type recommendation model is determined according to the processing effect or score of the corresponding historical task), and then the calculation method corresponding to the task type is selected, that is, the calculation method with the best effect, to avoid wasting privacy computing resources.

[0105] In this embodiment, when it is detected that a first participant participating in federated learning initiates a task to be processed, task information and a first data set associated with the task to be processed and sent by the first participant are received.

[0106] In this embodiment, the task to be processed can be an image (modeling) processing task based on federated learning, a recommendation (prediction) processing task based on federated learning, a classification processing task based on federated learning, a query based on federated learning, etc., which are not specifically limited here.

[0107] In this embodiment, it is a pending task submitted by a first participant, and the first participant is part of the hybrid architecture system (the hybrid architecture system also includes various subsystems, corresponding to multiple other second participants of the subsystems, and a third party), or the first participant is not part of the hybrid architecture system, but communicates with the hybrid architecture system.

[0108] When it is detected that the first participant participating in federated learning initiates a task to be processed, task information and information of the first data set associated with the task to be processed sent by the first participant are received, wherein the task information may be task type, task name, task initiation time, participants involved in the task, algorithm type involved in the task, etc., wherein the algorithm type may be K nearest neighbor algorithm, decision tree, K-Means clustering, neural network, etc.

[0109] Specifically, the first data set may be user data of a user corresponding to the first participant, such as a bank, or user data of a user corresponding to an e-commerce platform, which is not specifically limited here.

[0110] The information of the first data set may be the data size, data name, etc. of the first data set.

[0111] Step S20, receiving information of a second data set associated with the task to be processed, which is sent by the second participant after detecting the task to be processed;

[0112] In this embodiment, the first participant, as the initiator, initiates the task to be processed, and at this time, the other (associated) second participants detect the task to be processed.

[0113] When other second participants detect the task to be processed, they send information of the second data set associated with the task to be processed.

[0114] Specifically, the second data set may be user data of a user corresponding to the second participant, such as a bank, or user data of a user corresponding to an e-commerce platform, which is not specifically limited here.

[0115] The information of the first data set may be the data size, data name, etc. of the second data set.

[0116] In this embodiment, the specific content of the second data set is determined by the task to be processed.

[0117] Specifically, if the type of task to be processed is a vertical federation type, if the first data set is the user data of the first participant such as the user corresponding to the bank, then the second participant is the user data of the user corresponding to the e-commerce platform, where the bank and the e-commerce platform have many of the same users in the same region, but the business of the bank and the e-commerce platform is different.

[0118] If the type of the task to be processed is a horizontal federation type, the first data set may be user data of a user corresponding to a first participant, such as a first bank, and the second data set may be user data of a user corresponding to a second participant, such as a second bank in another place. The first bank and the second bank have the same business in different regions, but different users.

[0119] Step S30, determining the task type of the task to be processed according to the task information, the information of the first data set, the information of the second data set, and a preset task type recommendation model, wherein the task type includes a decentralized type and a centralized type;

[0120] In this embodiment, after determining the task information, the information of the first data set, and the information of the second data set, a preset task type recommendation model within a third party is called, or a preset task type recommendation model outside a third party is called, and the task information, the information of the first data set, and the information of the second data set are input into the preset task type recommendation model, and the task information, the information of the first data set, and the information of the second data set are predicted and processed based on the preset task type recommendation model to obtain the task type of the task to be processed.

[0121] In this embodiment, it should be clarified that the preset task type recommendation model is obtained through training. Specifically, the preset task type recommendation model is trained based on the calculation effect data of historical tasks under different task type corresponding modes. The preset task type recommendation model can accurately determine the predicted effect score of the task to be processed under different task type modes, and then determine the task type of the task to be processed based on the predicted effect score.

[0122] In this embodiment, it should be clarified that the task types include distributed type and centralized type, wherein the data scheduling mode corresponding to the distributed type task is the distributed scheduling mode, and the data scheduling mode corresponding to the centralized type task is the centralized scheduling mode.

[0123] Among them, the distributed scheduling mode: the original data is located in the local server of each participant. After the participants complete the calculation locally, they exchange the ciphertext data of the intermediate results through the network, such as Figure 5 as well as Figure 6 shown.

[0124] Data centralized mode: Each participant transmits the ciphertext of the original data to a centralized computing environment (third party). After the centralized computing environment completes the calculation, it returns the result to the task initiator, i.e. the first participant. Figure 4 shown.

[0125] Of course, the task type also includes other types, and the corresponding scheduling mode may also include other scheduling modes, which are not specifically limited here.

[0126] In this embodiment, the third party accurately determines the task type of the task to be processed, and then completes the processing of the task to be processed.

[0127] In this embodiment, the processing of the corresponding types of tasks to be processed can be implemented in a targeted manner.

[0128] Reference Figure 2 Step S30, determining the task type of the task to be processed according to the task information, the information of the first data set, the information of the second data set and a preset task type recommendation model, comprises:

[0129] Step S31, inputting the task information, the information of the first data set and the information of the second data set into a preset task type recommendation model;

[0130] The preset task type recommendation model is obtained by iterative training based on a preset training data set with task effect score labels;

[0131] In this embodiment, it is specifically described how to determine the task type of the task to be processed based on a preset task type recommendation model. The preset task type recommendation model is obtained by iterative training based on a preset training data set with a task effect score label, and the task effect score label can be a label such as excellent, good, or unsatisfactory, or the task effect score label is directly a score label.

[0132] Step S32, performing prediction processing on the task information, the information of the first data set, and the information of the second data set based on the preset task type recommendation model to obtain a prediction effect score;

[0133] Based on the preset task type recommendation model, the task information, the information of the first data set and the information of the second data set are predicted and processed to obtain a prediction effect score. Specifically, the preset task type recommendation model first converts the task information, the information of the first data set and the information of the second data set into a matrix, and then converts them into a matrix vector accordingly. Then, based on predetermined model parameters such as weight parameters, feature selection (vector operation process) is performed on the matrix vector to obtain a prediction effect score under different task types.

[0134] Before the step S32 of performing prediction processing on the task information, the information of the first data set, and the information of the second data set based on the preset task type recommendation model to obtain a prediction effect score, the method includes:

[0135] Step A1, obtaining a preset training data set with a task effect score label and a model to be trained, wherein the preset training data set has at least characteristic factors of task type, algorithm type and data attribute;

[0136] Step A2, determining a training matrix of the preset training data set based on the characteristic factors of the task type, algorithm type and data attributes;

[0137] In this embodiment, it is explained how to train and obtain a preset task type recommendation model. Specifically, first, a task effect score label processing is performed on the preset training data set. The purpose of the task effect score label processing is to verify whether the iteratively trained model meets the requirements.

[0138] In this embodiment, the preset training data set has at least characteristic factors of task information, algorithm type and data attribute (such as data type, data corresponding to specific content). Based on the characteristic factors of the task information, algorithm type and data attribute, a training matrix of the preset training data set is determined (based on the training matrix, a training vector is determined).

[0139] Step A3, inputting the training matrix into the model to be trained, and performing prediction processing on the training matrix based on the model to be trained to obtain a prediction result;

[0140] The training matrix (training vector) is input into the model to be trained, and prediction processing is performed on the training matrix based on the model to be trained (in the initial prediction process, parameters such as weights are randomly initialized) to obtain a prediction result.

[0141] Step A4, comparing the prediction result with the task type label and the task effect score label respectively, and obtaining a comparison result;

[0142] The type label in the prediction result is first compared with the task type label. After the comparison, the task effect score in the prediction result is obtained and compared with the task effect score label to obtain the comparison result.

[0143] Step A5, based on the comparison result, determining whether the model to be trained has completed training;

[0144] If the comparison result does not meet the training completion condition, it is determined that the model to be trained has not completed training.

[0145] Step A6, if the training is not completed, adjust the parameters of the model to be trained, return to input the training matrix into the model to be trained, perform prediction processing on the training matrix based on the model to be trained, and obtain the prediction result, until a preset task type recommendation model that meets the preset training completion conditions is obtained.

[0146] If the training is not completed, adjust the parameters of the model to be trained, that is, specifically, iteratively train the model parameters in the model to be trained until a preset task type recommendation model that meets the preset training completion condition is obtained, wherein the preset training completion condition may be when the number of training times reaches a preset number of times, or when the preset loss function converges.

[0147] Step S33: determining the task type of the task to be processed based on the prediction effect score.

[0148] In this embodiment, after the corresponding prediction effect score is obtained, the task type of the task to be processed is determined based on the prediction effect score.

[0149] Overall, taking an example to illustrate, when the task type is a decentralized type, if the predicted effect score is 0.8 points, and when the task type is a centralized type, if the predicted effect score is 0.5 points, the task type of the task to be processed is a decentralized type.

[0150] After determining the task type of the task to be processed, execute step S40, return the task type to the first participant and the second participant, and use a centralized scheduling mode to schedule centralized tasks / a decentralized scheduling mode to schedule decentralized type tasks, cooperate with the first participant and the second participant to perform privacy calculation on the task to be processed according to the target scheduling mode corresponding to the task type, and obtain a first processing result.

[0151] After obtaining the task type, the task type is returned to the first participant and the second participant. When the task to be processed is of a centralized type, the first participant sends the first data set to a third party, and the second participant sends the second data set to the third party. The third party combines the first data set and the second data set to perform centralized calculations. When the task to be processed is of a distributed type, the first participant, the second participant and the third party perform distributed calculations in coordination with the third party and other participants through local task scheduling.

[0152] After processing the task to be processed (privacy computing), a first processing result is obtained, wherein the first processing result can be a model that can be used to predict accuracy.

[0153] The present application provides a hybrid architecture system and a task scheduling method based on historical task effects. In the present application, when it is detected that a first participant participating in federated learning initiates a task to be processed, task information of the task to be processed and information of a first data set associated with the task to be processed sent by the first participant are received; information of a second data set associated with the task to be processed sent by the second participant after detecting the task to be processed is received; the task type of the task to be processed is determined according to the task information, the information of the first data set, the information of the second data set, and a preset task type recommendation model, wherein the task type includes a decentralized type and a centralized type; the task type is returned to the first participant and the second participant, and by adopting a centralized scheduling mode to schedule centralized tasks / adopting a decentralized scheduling mode to schedule decentralized type tasks, the first participant and the second participant cooperate to perform privacy calculation on the task to be processed according to the target scheduling mode corresponding to the task type, and obtain a first processing result. In the present application, a process for determining the task type corresponding to the task to be processed is added based on task information, information of the first data set, information of the second data set, and a preset task type recommendation model, and then task processing is performed on the task to be processed based on the corresponding scheduling modes of different task types (distributed type or centralized type). It can be understood that the specific calculation method is not selected by humans based on their own assumptions, but the task type corresponding to the task to be processed is clarified through the preset task type recommendation model (and the preset task type recommendation model is determined based on the processing effect or score of the corresponding historical task), and then the calculation method corresponding to the task type is selected, that is, the calculation method with the best effect, to avoid wasting privacy computing resources.

[0154] Embodiment 2

[0155] Further, based on the first embodiment of the present application, another embodiment of the present application is provided. In this embodiment, the task scheduling method based on historical task effects further includes:

[0156] Step S50, determining the number of tasks that have been scheduled in the historical stage;

[0157] In this embodiment, the number of tasks scheduled by the third party in the historical stage is also determined. The purpose of determining the number of tasks scheduled by the third party in the historical stage is to avoid inaccurate prediction caused by a small number of tasks processed in the historical stage.

[0158] Step S60, if the number of the scheduled tasks is less than the preset number, reading the default task type from the configuration information of the system, wherein the preset number is obtained by configuration, and the default task type includes a decentralized type and a centralized type;

[0159] In this embodiment, if the number of tasks scheduled by the third party is less than a preset number, such as less than 100 (the preset number is configured), a default task type is read from the system configuration information, and the default task type includes a decentralized type and a centralized type.

[0160] That is, in this embodiment, the task type of the task to be processed is directly manually configured. Since there are too few tasks scheduled by a third party, at this time, the processing efficiency of the task to be processed is higher based on manual configuration, that is, configuration based on human experience.

[0161] Step S70, determining a target scheduling mode corresponding to the default task type, so as to process the to-be-processed task based on the target scheduling mode and obtain a second processing result.

[0162] After configuring the task type, the third party determines a target scheduling mode corresponding to the configured task type, so as to process the to-be-processed task based on the target scheduling mode and obtain a second processing result.

[0163] In this embodiment, the number of tasks scheduled by the third party in the historical stage is determined;

[0164] If the number of the scheduled tasks is less than the preset number, determine the configured task type of the task to be processed, wherein the preset number is obtained by configuration, and the task type includes a decentralized type and a centralized type; determine the target scheduling mode corresponding to the configured task type, so as to process the task to be processed based on the target scheduling mode, and obtain a second processing result. In this embodiment, the situation where the prediction is inaccurate due to the small number of tasks processed in the historical stage is avoided.

[0165] Embodiment 3

[0166] Further, based on the first embodiment and the second embodiment of the present application, another embodiment of the present application is provided. In this embodiment, in step S40, the centralized task is scheduled by adopting a centralized scheduling mode, and the first participant and the second participant perform privacy calculation on the task to be processed according to the target scheduling mode corresponding to the task type, and obtain the first processing result, including:

[0167] Step B1, determining whether a first data set of a first participant and a second data set of a second participant are stored locally;

[0168] Step B2: if not, schedule the first participant to encrypt and upload the first data set, and schedule other second participants to encrypt and upload the second data set, and save the first data set and the second data set;

[0169] Step B3, retrieving the first data set and the second data set locally, performing centralized computing processing on the task to be processed, and obtaining a first processing result.

[0170] In this embodiment, a centralized scheduling mode is adopted to schedule centralized tasks, and a specific description is given by taking two participants as an example. The specific process may be:

[0171] The third party determines whether the first data set of the first participant and the second data set of the second participant are stored locally. If not, the first participant uploads the first data set and the second participant uploads the second data set to the third party.

[0172] Or the third party schedules the first participant to encrypt and upload the first data set, and schedules other second participants to encrypt and upload the second data set, and saves the first data set and the second data set. That is, the third party loads data from different parties and saves them (which can be done when bandwidth is idle).

[0173] During the task execution process, the third party retrieves the first data set and the second data set from the local, performs task calculation processing on the task to be processed, and obtains a first processing result, that is, the centralized computing environment performs calculations completely based on the local data of the centralized computing environment, which saves the transmission of intermediate data and has relatively low bandwidth requirements.

[0174] The centralized computing environment encrypts the first processing result and sends it to participant 1, and participant 1 receives and decrypts the encrypted first processing result.

[0175] Task execution process, such as Figure 4 shown.

[0176] In this embodiment, it is determined whether the first data set of the first participant and the second data set of the second participant are stored locally; if not, the first participant is scheduled to encrypt and upload the first data set, and other second participants are scheduled to encrypt and upload the second data set, and the first data set and the second data set are saved; the first data set and the second data set are retrieved locally, and the task to be processed is processed in a centralized manner to obtain a first processing result. In this embodiment, the calculation and processing of the task to be processed in a centralized mode is realized by the third party.

[0177] Embodiment 4

[0178] Further, based on the first embodiment, the second embodiment and the third embodiment of the present application, another embodiment of the present application is provided. In this embodiment, in step S40, the method of scheduling the distributed type tasks by adopting a distributed scheduling mode, and cooperating with the first participant and the second participant to perform privacy calculation on the to-be-processed task according to the target scheduling mode corresponding to the task type, and obtaining the first processing result includes:

[0179] Step C1, if the decentralized type of the task to be processed is a horizontal federated decentralized type, receiving first intermediate data encrypted and sent by each participant;

[0180] The first intermediate data is generated by each participant based on the corresponding first model gradient, and the first model gradient is obtained by each participant performing privacy calculation based on the corresponding local model;

[0181] In this embodiment, when the distributed scheduling mode is adopted to schedule the distributed type tasks, the type of the task to be processed is first determined, which can be a horizontal federation distributed type or a vertical federation distributed type.

[0182] Step C2, aggregating the first intermediate data to obtain aggregated intermediate data;

[0183] Step C3, sending the aggregated intermediate data to each participant, so that each participant updates their own model parameters based on the aggregated intermediate data, obtains their own updated local models, and performs privacy calculation based on their own updated local models to continue to generate new first intermediate data;

[0184] Step C4, iteratively returns to the step of receiving the first intermediate data sent by each participant until the first preset completion condition is met, and the data result corresponding to the first preset completion condition is used as the first processing result.

[0185] Specifically, combined Figure 5 Describe the process of scheduling horizontal federated distributed type tasks by adopting a distributed scheduling mode:

[0186] (1) Participant 1 initiates the task as the initiator. Each participant calculates the model gradient (first model gradient) locally and uses encryption technologies such as homomorphic encryption, differential privacy, or secret sharing to mask the gradient information. The masked results are sent to the aggregation server (third party).

[0187] (2) The aggregation server (third party) performs secure aggregation operations, such as using weighted averaging based on homomorphic encryption, to obtain the aggregated result (aggregated intermediate data);

[0188] (3) The server sends the aggregated results (aggregated intermediate data) to each participant;

[0189] (4) Each participant decrypts the received gradient and uses the decrypted gradient result to update its own model parameters to continue to generate the first intermediate data.

[0190] Iterate steps (1)-(4) until a first preset completion condition is met, and use the data result corresponding to the first preset completion condition as the first processing result.

[0191] In this embodiment, the first preset completion condition is that the number of training times reaches the first preset number of times, or the first preset loss function converges.

[0192] In this embodiment, the data result corresponding to the first preset completion condition is the model obtained after training.

[0193] In this embodiment, after the task to be processed is processed (the privacy computing reaches the first preset completion condition), a first processing result is obtained, wherein the first processing result can be a model obtained after horizontal federation, which can be used to predict accuracy.

[0194] In this embodiment, the most effective computing method is used to calculate the distributed tasks of the horizontal federation type, avoiding the waste of privacy computing resources.

[0195] In addition, in step S40, the method of scheduling the distributed type tasks by adopting the distributed scheduling mode, and cooperating with the first participant and the second participant to perform privacy calculation on the to-be-processed task according to the target scheduling mode corresponding to the task type, and obtaining the first processing result may also include:

[0196] Step D1, if the decentralized type of the task to be processed is a vertical federation type, receiving the intermediate results sent by each participant after encrypting and masking the second intermediate data;

[0197] The second intermediate data is generated by each participant based on the corresponding encrypted second model gradient exchange, and the second model gradient is calculated by each participant based on the corresponding local model;

[0198] Step D2, decrypting the intermediate result, and sending the decrypted intermediate result to the corresponding participant, so that each participant can update their own model parameters based on the intermediate result, obtain their own updated local models, and continue to generate new intermediate results based on their own updated local models;

[0199] Step D3, iteratively returns to the step of receiving the intermediate results sent by each participant after encrypting and masking the second intermediate data, until the second preset completion condition is met, and the data result corresponding to the second preset completion condition is used as the first processing result.

[0200] Specifically, Figure 6 As shown in the figure, the process of scheduling vertical federated distributed type tasks by adopting a distributed scheduling mode is described:

[0201] (1) Participant 1 initiates the task as the initiator, and the third party (coordinator) creates a key pair and sends the public key to Participant 1 and Participant 2;

[0202] (2) Participant 1 and Participant 2 encrypt and exchange the intermediate processing results, which are used to help calculate the gradient and loss value;

[0203] (3) Party 1 and Party 2 calculate the encrypted gradient and add the additional mask respectively to obtain the intermediate result (Party 2 also calculates the encryption loss). Party 1 and Party 2 send the encrypted result to the third party (coordinator);

[0204] (4) The third party (coordinator) decrypts the gradient and loss information and sends the results back to Party 1 and Party 2. Party 1 and Party 2 remove the mask on the gradient information and update the model parameters based on the gradient information.

[0205] In this embodiment, the second preset completion condition is that the number of training times reaches the second preset number of times, or the second preset loss function converges.

[0206] In this embodiment, after processing the task to be processed (the privacy computing reaches the second preset completion condition), a second processing result is obtained, wherein the first processing result can be a model obtained after vertical federation, which can be used to predict accuracy.

[0207] It can be understood that in this embodiment, the most effective computing method is used to calculate the distributed tasks of the vertical federation type, avoiding the waste of privacy computing resources.

[0208] Embodiment 5

[0209] Further, another embodiment of the present application is provided. In this embodiment, the method is applied to any first participant initiating a task to be processed in a participant subsystem, wherein the participant subsystem further includes at least one second participant associated with the task to be processed. The method for scheduling tasks based on historical task effects includes:

[0210] Step E1, when initiating a task to be processed, determining task information of the task to be processed and information of a first data set associated with the task to be processed, and sending the task information and information of the first data set to a third party;

[0211] Step E2, receiving the task type of the task to be processed sent by the third party, and when the task to be processed is of a centralized type, sending the first data set to the third party, so that the third party can perform centralized computing in combination with the first data set and the second data set uploaded by the second participant after receiving the task type, to obtain a first processing result; or, when the task to be processed is of a decentralized type, performing decentralized computing in coordination with the third party and other participants through local task scheduling, to obtain the first processing result;

[0212] Among them, the task type is determined by the third party based on the task information, the information of the first data set, the information of the second data set and a preset task type recommendation model, the task type includes a decentralized type and a centralized type, the information of the second data set is sent to the third party by the second participant after detecting the task to be processed, and the second data set is associated with the task to be processed.

[0213] In this embodiment, the execution subject is any first participant in the participant subsystem who initiates the task to be processed. When the first participant is in an idle state, the first local data set is uploaded to the third party. The specific implementation method of the first participant for task scheduling is basically the same as the above-mentioned embodiments of the task scheduling method based on historical task effects, and will not be repeated here.

[0214] Embodiment 6

[0215] Furthermore, based on the fifth embodiment of the present application, another embodiment of the present application is provided. In this embodiment, after the processing of the task to be processed is completed, the task processing process of the task to be processed is scored based on the processing effect, and the task information, task type and the score of the task to be processed are sent to the task type recommendation module of the third party accordingly, so that the third party stores the task information, the task type, the score of the task to be processed, and the first data set information and the second data set information associated with the task to be processed accordingly based on the task information.

[0216] In this embodiment, the processing effect may be processing time consumption or processing resource consumption. Since in this embodiment, the task processing process of the task to be processed is also scored, it is convenient to select the best privacy computing method in the subsequent privacy computing process.

[0217] Embodiment 7

[0218] Further, another embodiment of the present application is provided. In this embodiment, the method for scheduling tasks based on historical task effects is applied to any second participant in a participant subsystem that is passively associated with a task to be processed, and the participant subsystem also includes a first participant that initiates the task to be processed, and includes:

[0219] When a task to be processed is monitored, determining information of a second data set associated with the task to be processed, and sending the information of the second data set to a third party;

[0220] receiving the task type of the task to be processed sent by the third party, and when the task to be processed is of a centralized type, sending the second data set to the third party, so that the third party can perform centralized computing on the basis of the second data set and the first data set uploaded by the first participant after receiving the task type, to obtain a first processing result; or, when the task to be processed is of a decentralized type, performing decentralized computing in cooperation with the third party and other participants through local task scheduling, to obtain the first processing result;

[0221] Among them, the task type is determined by the third party based on the task information, the information of the first data set, the information of the second data set and a preset task type recommendation model, the task type includes a decentralized type and a centralized type, the information of the first data set is sent to the third party by the first participant after initiating the task to be processed, and the first data set is associated with the task to be processed.

[0222] In this embodiment, the executing entity is the second participant in the participant subsystem. When the second participant is in an idle state, the second participant uploads the local second data set to the third party. The specific implementation method of task scheduling by the second participant is basically the same as the above-mentioned embodiments of the task scheduling method based on historical task effects, and will not be repeated here.

[0223] Embodiment 8

[0224] Further, based on the fifth embodiment, the sixth embodiment and the seventh embodiment of the present application, another embodiment of the present application is provided. In this embodiment, when the task to be processed is of a distributed type, the step of obtaining a first processing result by cooperating with a third party and other participants to perform distributed computing through local task scheduling includes:

[0225] If the decentralized type of the task to be processed is a horizontal federation type, sending first intermediate data to the third party; wherein the first intermediate data is generated by each participant based on a corresponding first model gradient, and the first model gradient is calculated by each participant based on a corresponding local model;

[0226] Receiving aggregated intermediate data sent by a third party, wherein the aggregated intermediate data is obtained by the third party aggregating the first intermediate data;

[0227] Updating model parameters based on the aggregated intermediate data to obtain updated local models, and continuing to generate new first intermediate data based on the respective updated local models;

[0228] Iterate and return to the step of sending the first intermediate data to the third party until a first preset completion condition is met, and use the data result corresponding to the first preset completion condition as the first processing result.

[0229] The specific implementation method of task scheduling in this embodiment is basically the same as the above-mentioned embodiments of the task scheduling method based on historical task effects, and will not be repeated here.

[0230] Embodiment 9

[0231] Further, based on the fifth, sixth, seventh and eighth embodiments of the present application, another embodiment of the present application is provided. In this embodiment, when the task to be processed is of a distributed type, the step of obtaining a first processing result by cooperating with a third party and other participants to perform distributed computing through local task scheduling includes:

[0232] If the decentralized type of the task to be processed is a vertical federation type, encrypting and masking the second intermediate data to obtain an intermediate result, and sending the intermediate result to the third party;

[0233] The second intermediate data is generated by each participant based on the corresponding encrypted second model gradient exchange, and the second model gradient is calculated by each participant based on the corresponding local model;

[0234] After receiving the decrypted intermediate result sent by the third party after decryption processing of the intermediate result, updating the respective model parameters based on the decrypted intermediate result, and obtaining the respective updated local models, and continuing to generate new intermediate results based on the respective updated local models;

[0235] Iterate and return to the step of encrypting the second intermediate data and obtaining the intermediate result after masking, until the second preset completion condition is met, and take the data result corresponding to the second preset completion condition as the first processing result.

[0236] The task scheduling method in this embodiment is basically the same as the above-mentioned embodiments of the task scheduling method based on historical task effects, and will not be described in detail here.

[0237] Embodiment 10

[0238] Further, based on all the above embodiments, another embodiment of the present application is provided. In this embodiment, a hybrid architecture system is provided, and the hybrid architecture system includes:

[0239] A first participant, the first participant is used to: initiate a task to be processed, and send task information of the task to be processed and information of a first data set associated with the task to be processed to a third party; and also to send the first data set to the third party when the task to be processed is of a centralized type, or to coordinate with the third party and other participants to perform distributed computing through local task scheduling when the task to be processed is of a distributed type;

[0240] a second participant, the second participant being used to: after detecting that the first participant has initiated a task to be processed, send information of a second data set associated with the task to be processed to a third party; and also being used to send the second data set to the third party when the task to be processed is of a centralized type, or to coordinate with the third party and other participants to perform distributed computing through local task scheduling when the task to be processed is of a distributed type;

[0241] A third party, the third party is used to: determine the task type of the task to be processed based on the task information, the first data set information and the second data set information, wherein the task type includes a decentralized type and a centralized type; and is also used to perform centralized computing in combination with the first data set and the second data set when the task to be processed is of a centralized type, or to perform decentralized computing through local computing in coordination with other participants when the task to be processed is of a decentralized type.

[0242] There may be multiple second parties.

[0243] The specific implementation of the hybrid architecture system of the present application is basically the same as the various embodiments of the task scheduling method based on historical task effects described above, and will not be repeated here.

[0244] In this embodiment, the first participant includes a first scheduling module, a first computing module and a first communication module. The first scheduling module is used to send task information of the task to be processed and information of a first data set associated with the task to be processed to a third party through the first communication module after initiating the task to be processed, and is also used to schedule the first computing module to perform local calculations through local tasks to cooperate with the third party and other participants to perform distributed calculations when the task to be processed is a distributed type, or to send the first data set to the third party through the first communication module when the task to be processed is a centralized type.

[0245] In this embodiment, the first participant includes a first scheduling module, a first calculation module and a first communication module. In addition, the first participant may also include a first storage module.

[0246] In this embodiment, the first scheduling module is used to initiate tasks to be processed (including modeling, prediction, query and other tasks), and to obtain task information of the tasks to be processed and information of a first data set associated with the tasks to be processed, and to send the task information of the tasks to be processed and information of the first data set associated with the tasks to be processed to a third party through the first communication module; when the tasks to be processed are of a centralized type, the first data set is sent to the third party through the first communication module.

[0247] In this embodiment, the first communication module is used to receive the task information of the task to be processed and the information of the first data set associated with the task to be processed sent by the first scheduling module, and send the task information of the task to be processed and the information of the first data set associated with the task to be processed to a third party. When the task to be processed is of a centralized type, the first data set sent by the first scheduling module is received, and the first data set is sent to a third party.

[0248] In this embodiment, the first computing module is used to send the first intermediate data to the third party if the decentralized type of the task to be processed is a horizontal federation type; wherein the first intermediate data is generated by each participant based on the corresponding first model gradient, and the first model gradient is calculated by each participant based on the corresponding local model; the first computing module is also used to receive aggregated intermediate data sent by the third party, wherein the aggregated intermediate data is obtained by aggregating the first intermediate data by the third party; the first computing module is also used to update the model parameters based on the aggregated intermediate data to obtain an updated local model, and continue to generate new first intermediate data based on the respective updated local models; the first computing module is also used to iteratively return to the step of sending the first intermediate data to the third party until the first preset completion condition is met, and the data result corresponding to the first preset completion condition is used as the first processing result.

[0249] In this embodiment, the first computing module is used to, if the decentralized type of the task to be processed is a vertical federation type, encrypt and mask the second intermediate data to obtain an intermediate result, and send the intermediate result to the third party; wherein, the second intermediate data is generated by each participant based on the corresponding encrypted second model gradient exchange, and the second model gradient is calculated by each participant based on the corresponding local model; the first computing module is also used to receive the decrypted intermediate result sent by the third party after decrypting the intermediate result, update the respective model parameters based on the decrypted intermediate result, obtain the respective updated local models, and continue to generate new intermediate results based on the respective updated local models; the first computing module is also used to iteratively return to the step of encrypting and masking the second intermediate data to obtain the intermediate result until the second preset completion condition is met, and the data result corresponding to the second preset completion condition is used as the first processing result.

[0250] The first storage module is used to store task information of the task to be processed, information of the first data set of the task to be processed, and the first data set.

[0251] In this embodiment, the second participant includes a second scheduling module, a second computing module and a second communication module. The second scheduling module is used to: after detecting that the first participant initiates a task to be processed, send information of the second data set associated with the task to be processed to a third party through the second communication module, and is also used to schedule the second computing module to perform local calculations through local tasks to cooperate with the third party and other participants to perform distributed calculations when the task to be processed is a distributed type, or to send the second data set to the third party through the second communication module when the task to be processed is a centralized type.

[0252] In this embodiment, the second participant includes a second scheduling module, a second calculation module and a second communication module. In addition, the second participant may also include a second storage module.

[0253] In this embodiment, the second scheduling module is used to obtain information of a second data set associated with the task to be processed after detecting that the first participant has initiated the task to be processed, and send the information of the second data set associated with the task to be processed to a third party through the second communication module; when the task to be processed is of a centralized type, the second data set is sent to the third party through the second communication module.

[0254] In this embodiment, the second communication module is used to receive information of the second data set associated with the task to be processed sent by the second scheduling module, and send the task information of the task to be processed and the information of the second data set associated with the task to be processed to a third party. When the task to be processed is of a centralized type, the second data set sent by the second scheduling module is received, and the second data set is sent to a third party.

[0255] In this embodiment, the second computing module is used to send the first intermediate data to the third party if the decentralized type of the task to be processed is a horizontal federation type; wherein the first intermediate data is generated by each participant based on the corresponding first model gradient, and the first model gradient is calculated by each participant based on the corresponding local model; the second computing module is also used to receive aggregated intermediate data sent by the third party, wherein the aggregated intermediate data is obtained by aggregating the first intermediate data by the third party; the second computing module is also used to update the model parameters based on the aggregated intermediate data to obtain an updated local model, and continue to generate new first intermediate data based on the respective updated local models; the second computing module is also used to iteratively return to the step of sending the first intermediate data to the third party until the first preset completion condition is met, and the data result corresponding to the first preset completion condition is used as the first processing result.

[0256] In this embodiment, the second computing module is used to encrypt and mask the second intermediate data to obtain an intermediate result if the decentralized type of the task to be processed is a vertical federation type, and send the intermediate result to the third party; wherein the second intermediate data is generated by each participant based on the corresponding encrypted second model gradient exchange, and the second model gradient is calculated by each participant based on the corresponding local model; the second computing module is also used to receive the decrypted intermediate result sent by the third party after decrypting the intermediate result, update the respective model parameters based on the decrypted intermediate result, obtain the respective updated local models, and continue to generate new intermediate results based on the respective updated local models; the second computing module is also used to iteratively return to the step of encrypting and masking the second intermediate data to obtain the intermediate result until the second preset completion condition is met, and the data result corresponding to the second preset completion condition is used as the first processing result.

[0257] The second storage module is used to store the task information of the task to be processed, the information of the second data set of the task to be processed, and the second data set.

[0258] In this embodiment, the third party includes a third scheduling module, a third computing module, a third communication module and a task type recommendation module, the third communication module is used to communicate with the first participant and / or the second participant; the task type recommendation module is used to: determine the task type of the task to be processed based on the task information, the information of the first data set and the information of the second data set; the third scheduling module is used to: when the task to be processed is of a centralized type, schedule the third computing module to perform centralized calculations in combination with the first data set and the second data set, or when the task to be processed is of a distributed type, schedule the third computing module to perform local calculations through local tasks to cooperate with other participants to perform distributed calculations.

[0259] In this embodiment, the third party includes a third scheduling module, a third calculation module, a third communication module and a task type recommendation module. In addition, the second participant may also include a third storage module.

[0260] In this embodiment, the third computing module is used to receive the first intermediate data encrypted and sent by each participant if the decentralized type of the task to be processed is a horizontal federated decentralized type; wherein the first intermediate data is generated by each participant based on the corresponding first model gradient, and the first model gradient is obtained by each participant performing privacy calculation based on the corresponding local model; the third computing module is also used to aggregate the first intermediate data to obtain aggregated intermediate data; the aggregated intermediate data is sent to each participant so that each participant can update their respective model parameters based on the aggregated intermediate data, obtain their respective updated local models, and perform privacy calculation based on their respective updated local models to continue to generate new first intermediate data; the third computing module is also used to iteratively return to the step of receiving the first intermediate data sent by each participant until the first preset completion condition is met, and the data result corresponding to the first preset completion condition is used as the first processing result.

[0261] In this embodiment, the third computing module is used to receive the intermediate results sent by each participant after encrypting and masking the second intermediate data if the decentralized type of the task to be processed is a vertical federation type; wherein the second intermediate data is generated by each participant based on the corresponding encrypted second model gradient exchange, and the second model gradient is calculated by each participant based on the corresponding local model; the intermediate result is decrypted and sent to the corresponding participant, so that each participant can update their own model parameters based on the intermediate result, and obtain their own updated local models, and continue to generate new intermediate results based on their own updated local models; iteratively return to the step of receiving the intermediate results sent by each participant after encrypting and masking the second intermediate data, until the second preset completion condition is met, and the data result corresponding to the second preset completion condition is used as the first processing result.

[0262] In this embodiment, the third computing module is further used to perform centralized computing processing on the task to be processed to obtain a first processing result.

[0263] Furthermore, in this embodiment, the first participant also includes a task scoring module, which is used to score the task processing process of the task to be processed according to the processing effect after the processing of the task to be processed is completed, and send the task information of the task to be processed and the score to the task type recommendation module of the third party; the task type recommendation module of the third party stores the task information, the task type, the score of the task to be processed, and the first data set information and the second data set information associated with the task to be processed based on the task information, for use by the third party when training a preset task type recommendation model.

[0264] Embodiment 11

[0265] An electronic device is provided, which is a physical node device, and includes: a memory, a processor, and a program stored in the memory for implementing the task scheduling method based on historical task effects, wherein the memory is used to store the program for implementing the task scheduling method; the processor is used to execute the program for implementing the task scheduling method based on historical task effects, so as to implement the steps of the task scheduling method in the above-mentioned embodiment.

[0266] Reference Figure 3 , Figure 3 It is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiment of the present application.

[0267] like Figure 3As shown, the electronic device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the processor 1001 and the memory 1005. The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0268] Optionally, the electronic device may also include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, etc. The rectangular user interface may include a display screen (Display), an input submodule such as a keyboard (Keyboard), and the optional rectangular user interface may also include a standard wired interface and a wireless interface. The network interface may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0269] Those skilled in the art will understand that Figure 3 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0270] like Figure 3 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module and a task scheduler. The operating system is a program that manages and controls the hardware and software resources of the electronic device, and supports the operation of the task scheduler and other software and / or programs. The network communication module is used to realize the communication between the components inside the memory 1005, and the communication with other hardware and software in the hybrid architecture system.

[0271] exist Figure 3 In the electronic device shown, the processor 1001 is used to execute the task scheduling program stored in the memory 1005 to implement the steps of any of the above-mentioned task scheduling methods based on historical task effects.

[0272] The specific implementation of the electronic device of the present application is basically the same as the various embodiments of the task scheduling method, and will not be repeated here.

[0273] Embodiment 12

[0274] An embodiment of the present application provides a storage medium, and the storage medium stores one or more programs, and the one or more programs can also be executed by one or more processors to implement the steps of the task scheduling method in the above embodiment.

[0275] The specific implementation method of the storage medium of the present application is basically the same as the above-mentioned embodiments of the task scheduling method, and will not be repeated here.

[0276] Embodiment 13

[0277] The present application also provides a computer program product, including a computer program, which implements the steps of the task scheduling method in the above embodiment when executed by a processor.

[0278] The specific implementation of the computer program product of the present application is basically the same as the various embodiments of the above-mentioned task scheduling method, and will not be repeated here.

[0279] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0280] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0281] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0282] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A hybrid architecture system, characterized in that: The hybrid architecture system includes: A first participant, the first participant is used to: initiate a task to be processed, and send task information of the task to be processed and information of a first data set associated with the task to be processed to a third party; and also to send the first data set to the third party when the task to be processed is of a centralized type, or to coordinate with the third party and other participants to perform distributed computing through local task scheduling when the task to be processed is of a distributed type; a second participant, the second participant being used to: after detecting that the first participant has initiated a task to be processed, send information of a second data set associated with the task to be processed to a third party; and also being used to send the second data set to the third party when the task to be processed is of a centralized type, or to coordinate with the third party and other participants to perform distributed computing through local task scheduling when the task to be processed is of a distributed type; A third party, the third party is used to: input the task information, the information of the first data set and the information of the second data set into a preset task type recommendation model; wherein the preset task type recommendation model is obtained by iterative training based on a preset training data set with a task effect score label; based on the preset task type recommendation model, predictive processing is performed on the task information, the information of the first data set and the information of the second data set to obtain a predicted effect score; based on the predicted effect score, the task type of the task to be processed is determined, wherein the task type includes a decentralized type and a centralized type; and is also used to perform centralized operations in combination with the first data set and the second data set when the task to be processed is a centralized type, or to perform decentralized operations through local calculations in cooperation with other participants when the task to be processed is a decentralized type.

2. The hybrid architecture system according to claim 1, characterized in that: The number of the second participants is multiple.

3. The hybrid architecture system according to claim 1, characterized in that: The first participant includes a first scheduling module, a first computing module and a first communication module. The first scheduling module is used to send task information of the task to be processed and information of a first data set associated with the task to be processed to a third party through the first communication module after initiating the task to be processed. It is also used to schedule the first computing module to perform local calculations through local tasks to cooperate with the third party and other participants to perform distributed calculations when the task to be processed is a distributed type, or to send the first data set to the third party through the first communication module when the task to be processed is a centralized type.

4. The hybrid architecture system according to claim 1, characterized in that: The second participant includes a second scheduling module, a second computing module and a second communication module. The second scheduling module is used to: after detecting that the first participant initiates a task to be processed, send information of a second data set associated with the task to be processed to a third party through the second communication module; and is also used to, when the task to be processed is of a distributed type, schedule the second computing module to perform local calculations through local tasks to cooperate with the third party and other participants to perform distributed calculations, or when the task to be processed is of a centralized type, send the second data set to the third party through the second communication module.

5. The hybrid architecture system according to claim 1, characterized in that: The third party includes a third scheduling module, a third computing module, a third communication module and a task type recommendation module, wherein the third communication module is used to communicate with the first participant and / or the second participant; the task type recommendation module is used to determine the task type of the task to be processed based on the task information, the information of the first data set and the information of the second data set; The third scheduling module is used to: when the task to be processed is of a centralized type, schedule the third computing module to perform centralized computing in combination with the first data set and the second data set; or when the task to be processed is of a distributed type, schedule the third computing module to perform local computing through local tasks to cooperate with other participants to perform distributed computing.

6. The hybrid architecture system according to claim 5, characterized in that: The first participant also includes a task scoring module, which is used to score the task processing process of the task to be processed according to the processing effect after the task to be processed is completed, and send the task information of the task to be processed and the score to the task type recommendation module of the third party; The task type recommendation module of the third party stores the task information, the task type, the score of the task to be processed, and the first data set information and the second data set information associated with the task to be processed accordingly based on the task information.

7. A task scheduling method based on historical task effects, characterized in that: Applied to a third party, the task scheduling method based on historical task effects includes: When detecting that a first participant participating in federated learning initiates a task to be processed, receiving task information of the task to be processed and information of a first data set associated with the task to be processed sent by the first participant; receiving information of a second data set associated with the task to be processed and sent by the second participant after detecting the task to be processed; Determine the task type of the task to be processed according to the task information, the information of the first data set, the information of the second data set, and a preset task type recommendation model, wherein the task type includes a decentralized type and a centralized type; The step of determining the task type of the task to be processed according to the task information, the information of the first data set, the information of the second data set, and a preset task type recommendation model includes: Inputting the task information, the information of the first data set and the information of the second data set into a preset task type recommendation model; The preset task type recommendation model is obtained by iterative training based on a preset training data set with task effect score labels; Based on the preset task type recommendation model, prediction processing is performed on the task information, the information of the first data set, and the information of the second data set to obtain a prediction effect score; Based on the prediction effect score, determining the task type of the task to be processed; The task type is returned to the first participant and the second participant, and the first participant and the second participant cooperate to perform privacy calculation on the task to be processed according to the target scheduling mode corresponding to the task type by adopting a centralized scheduling mode to schedule the centralized task / adopting a decentralized scheduling mode to schedule the decentralized type task, and obtain a first processing result.

8. The task scheduling method based on historical task effects according to claim 7 is characterized in that: Before the step of performing prediction processing on the task information, the information of the first data set, and the information of the second data set based on the preset task type recommendation model to obtain a prediction effect score, the method includes: Obtaining a preset training data set with a task effect score label and a model to be trained, wherein the preset training data set has at least characteristic factors of task type, algorithm type and data attribute; Determining a training matrix of the preset training data set based on the characteristic factors of the task type, algorithm type and data attributes; Inputting the training matrix into the model to be trained, and performing prediction processing on the training matrix based on the model to be trained to obtain a prediction result; Comparing the prediction result with the task effect score label and obtaining a comparison result; Based on the comparison result, determining whether the model to be trained has completed training; If the training is not completed, the parameters of the model to be trained are adjusted, and the training matrix is ​​returned to be input into the model to be trained, and the training matrix is ​​predicted based on the model to be trained to obtain the prediction result, until a preset task type recommendation model that meets the preset training completion conditions is obtained.

9. The task scheduling method based on historical task effects according to claim 7 is characterized in that: The task scheduling method based on historical task effects also includes: Determine the number of tasks that have been scheduled in the historical period; If the number of the scheduled tasks is less than a preset number, a default task type is read from the configuration information of the system, wherein the preset number is obtained by configuration, and the default task type includes a decentralized type and a centralized type; Determine a target scheduling mode corresponding to the default task type to process the to-be-processed task based on the target scheduling mode and obtain a second processing result.

10. The task scheduling method based on historical task effects according to claim 7, characterized in that: The method of scheduling the centralized task by adopting the centralized scheduling mode, cooperating with the first participant and the second participant to perform privacy calculation on the task to be processed according to the target scheduling mode corresponding to the task type, and obtaining the first processing result includes: Determine whether a first data set of the first participant and a second data set of the second participant are stored locally; If not, scheduling the first participant to encrypt and upload the first data set, and scheduling other second participants to encrypt and upload the second data set, and saving the first data set and the second data set; The first data set and the second data set are retrieved locally, and centralized computing is performed on the task to be processed to obtain a first processing result.

11. The task scheduling method based on historical task effects according to claim 7, characterized in that: The method of scheduling the distributed type tasks by adopting a distributed scheduling mode, and cooperating with the first participant and the second participant to perform privacy calculation on the to-be-processed tasks according to the target scheduling mode corresponding to the task type, and obtaining the first processing result, includes: If the decentralized type of the task to be processed is a horizontal federated decentralized type, receiving first intermediate data encrypted and sent by each participant; The first intermediate data is generated by each participant based on the corresponding first model gradient, and the first model gradient is obtained by each participant performing privacy calculation based on the corresponding local model; aggregating the first intermediate data to obtain aggregated intermediate data; Sending the aggregated intermediate data to each participant, so that each participant updates their own model parameters based on the aggregated intermediate data, obtains their own updated local models, and performs privacy calculations based on their own updated local models to continue to generate new first intermediate data; Iterate and return to the step of receiving the first intermediate data sent by each participant until the first preset completion condition is met, and take the data result corresponding to the first preset completion condition as the first processing result.

12. The task scheduling method based on historical task effects according to claim 7, characterized in that: The method of scheduling the distributed type tasks by adopting a distributed scheduling mode, and cooperating with the first participant and the second participant to perform privacy calculation on the to-be-processed tasks according to the target scheduling mode corresponding to the task type, and obtaining the first processing result, includes: If the decentralized type of the task to be processed is a vertical federation type, receiving the intermediate results sent by each participant after encrypting and masking the second intermediate data; The second intermediate data is generated by each participant based on the corresponding encrypted second model gradient exchange, and the second model gradient is calculated by each participant based on the corresponding local model; Decrypting the intermediate result, and sending the decrypted intermediate result to the corresponding participant, so that each participant can update their own model parameters based on the intermediate result, obtain their own updated local models, and continue to generate new intermediate results based on their own updated local models; Iterate and return to the step of receiving the intermediate results sent by each participant after encrypting and masking the second intermediate data, until the second preset completion condition is met, and the data result corresponding to the second preset completion condition is used as the first processing result.

13. A task scheduling method based on historical task effects, characterized in that: Applied to any first participant initiating a task to be processed in a participant subsystem, the participant subsystem further comprising at least one second participant associated with the task to be processed, the task scheduling method based on historical task effects comprises: When initiating a task to be processed, determining task information of the task to be processed and information of a first data set associated with the task to be processed, and sending the task information and information of the first data set to a third party; receiving the task type of the task to be processed sent by the third party, and when the task to be processed is of a centralized type, sending the first data set to the third party, so that the third party can perform centralized computing on the first data set and the second data set uploaded by the second participant after receiving the task type, to obtain a first processing result; or, when the task to be processed is of a decentralized type, performing decentralized computing in coordination with the third party and other participants through local task scheduling, to obtain the first processing result; Among them, the task type is that the third party inputs the task information, the information of the first data set and the information of the second data set into a preset task type recommendation model, wherein the preset task type recommendation model is obtained by iterative training based on a preset training data set with a task effect score label, and predicts the task information, the information of the first data set and the information of the second data set based on the preset task type recommendation model to obtain a predicted effect score, and then determines based on the predicted effect score that the task type includes a decentralized type and a centralized type, and the information of the second data set is sent to the third party by the second participant after detecting the task to be processed, and the second data set is associated with the task to be processed.

14. The task scheduling method based on historical task effects according to claim 13, characterized in that: Also includes: After the processing of the task to be processed is completed, the task processing process of the task to be processed is scored according to the processing effect, and the task information, task type and the score of the task to be processed are sent to the task type recommendation module of the third party accordingly, so that the third party stores the task information, the task type, the score of the task to be processed, and the first data set information and the second data set information associated with the task to be processed accordingly based on the task information.

15. A task scheduling method based on historical task effects, characterized in that: Applied to any second participant in a participant subsystem that is passively associated with a task to be processed, the participant subsystem also includes a first participant that initiates the task to be processed, the task scheduling method based on historical task effects includes: When a task to be processed is monitored, determining information of a second data set associated with the task to be processed, and sending the information of the second data set to a third party; receiving the task type of the task to be processed sent by the third party, and when the task to be processed is of a centralized type, sending the second data set to the third party, so that the third party can perform centralized computing on the basis of the second data set and the first data set uploaded by the first participant after receiving the task type, to obtain a first processing result; or, when the task to be processed is of a decentralized type, performing decentralized computing in cooperation with the third party and other participants through local task scheduling, to obtain the first processing result; Among them, the task type is that the third party inputs the task information, the information of the first data set and the information of the second data set into a preset task type recommendation model, wherein the preset task type recommendation model is obtained by iterative training based on a preset training data set with a task effect score label, and predicts the task information, the information of the first data set and the information of the second data set based on the preset task type recommendation model to obtain a predicted effect score, and then determines based on the predicted effect score, the task type includes a decentralized type and a centralized type, the information of the first data set is sent to the third party by the first participant after initiating the task to be processed, and the first data set is associated with the task to be processed.

16. The task scheduling method based on historical task effects according to claim 15, characterized in that: The task scheduling method based on historical task effects also includes: When in an idle state, the local first data set is uploaded to the third party.

17. The task scheduling method based on historical task effects according to claim 15, characterized in that: When the task to be processed is of a distributed type, the step of obtaining a first processing result by cooperating with a third party and other participants to perform distributed computing through local task scheduling includes: If the decentralized type of the task to be processed is a horizontal federation type, sending first intermediate data to the third party; wherein the first intermediate data is generated by each participant based on a corresponding first model gradient, and the first model gradient is calculated by each participant based on a corresponding local model; Receiving aggregated intermediate data sent by a third party, wherein the aggregated intermediate data is obtained by the third party aggregating the first intermediate data; updating model parameters based on the aggregated intermediate data to obtain updated local models, and continuing to generate new first intermediate data based on the respective updated local models; Iterate and return to the step of sending the first intermediate data to the third party until a first preset completion condition is met, and use the data result corresponding to the first preset completion condition as the first processing result.

18. The task scheduling method based on historical task effects according to claim 15, characterized in that: When the task to be processed is of a distributed type, the step of obtaining a first processing result by cooperating with a third party and other participants to perform distributed computing through local task scheduling includes: If the decentralized type of the task to be processed is a vertical federation type, encrypting and masking the second intermediate data to obtain an intermediate result, and sending the intermediate result to the third party; The second intermediate data is generated by each participant based on the corresponding encrypted second model gradient exchange, and the second model gradient is calculated by each participant based on the corresponding local model; After receiving the decrypted intermediate result sent by the third party after decryption processing of the intermediate result, updating the respective model parameters based on the decrypted intermediate result, and obtaining the respective updated local models, and continuing to generate new intermediate results based on the respective updated local models; Iterate and return to the step of encrypting the second intermediate data and obtaining the intermediate result after masking, until the second preset completion condition is met, and take the data result corresponding to the second preset completion condition as the first processing result.

19. An electronic device, characterized in that: The electronic device is a physical node device, and includes: a memory, a processor, and a program stored in the memory for implementing the task scheduling method based on historical task effects. The memory is used to store a program for implementing the task scheduling method; The processor is used to execute a program for implementing the task scheduling method based on historical task effects, so as to implement the steps of the task scheduling method based on historical task effects as claimed in any one of claims 7 to 18.

20. A storage medium, characterized in that: The storage medium stores a program for implementing the task scheduling method, and the program for implementing the task scheduling method is executed by a processor to implement the steps of the task scheduling method based on historical task effects as described in any one of claims 7 to 18.

21. A product, the product being a computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the task scheduling method based on historical task effects as described in any one of claims 7 to 18 are implemented.

Images