Joint learning engine architecture system for multitask parallelism

Abstract

The application relates to a multi-task parallel joint learning engine architecture system, comprising a client component and a cloud service component, wherein the client component comprises a training unit and a plurality of clients, which are respectively used for obtaining a local training result based on preset local model training target data and uploading the local training result to the cloud service component; the cloud service component comprises a synchronization service unit, a task processing library, a task consumption unit and a task control unit; the synchronization service unit is used for receiving and synchronizing the local training result to the task control unit; the task processing library is used for storing a plurality of joint learning tasks to obtain a task list; the task consumption unit is used for determining at least one to-be-executed joint learning task from the task list; the task control unit is used for executing the at least one to-be-executed joint learning task, and matching a best engine function module according to an execution result and the local training result to perform aggregation processing on the local training result, so as to update the task list. Therefore, the problem that a user needs to queue and wait due to too many created tasks is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of federated learning, and in particular to a multi-task parallel federated learning engine architecture system. BACKGROUND

[0002] With the development of artificial intelligence technology, federated learning engine systems have been widely used in the field of Internet of Things. As a distributed machine learning paradigm, it technically breaks the problem of data silos, cooperates different data owners without exchanging data by designing a virtual model, establishes a shared data strategy while protecting data privacy and meeting data specifications.

[0003] In related technologies, single-task learning is mainly used, and each module is repeatedly developed to support users to create multiple tasks.

[0004] However, through single-task learning and repeated development of each module to help users achieve multiple task requirements, the learning engine may have inconsistent interface standards, etc., so it cannot provide stable and reliable cloud infrastructure for cloud services, cannot fully guarantee the normal operation of cloud services, and thus cannot better system mobilize and manage multiple joint participants, thereby reducing the efficiency of federated learning. SUMMARY

[0005] The present application provides a multi-task parallel federated learning engine architecture system to solve the problem of queuing and waiting due to too many tasks created by users, poor system mobilization and management of multiple joint participants, and reduced federated learning efficiency.

[0006] The first aspect of the present application provides a multi-task parallel federated learning engine architecture system, comprising:

[0007] A client component and a cloud service component, wherein,

[0008] The client component comprises:

[0009] A training unit configured to train target data based on a preset local model to obtain a local training result;

[0010] A plurality of clients configured to upload the local training result to the cloud service component;

[0011] The cloud service component comprises:

[0012] A synchronization service unit configured to receive and synchronize the local training result to a task control unit;

[0013] A task processing library configured to store a plurality of federated learning tasks created based on registered resources to obtain a task list;

[0014] a task consumption unit configured to determine at least one joint learning task to be executed from the task list;

[0015] The task control unit is configured to execute the at least one joint learning task to be executed, match an optimal engine function module according to an execution result and the local training result, aggregate the local training result through the optimal engine function module, and update the task list according to an aggregation processing result.

[0016] According to an embodiment of the present application, the engine function module includes at least one of an aggregation strategy, a heterogeneous strategy optimization, a joint knowledge distillation, an adaptive mechanism, and an exception handling strategy.

[0017] According to an embodiment of the present application, the task control unit is provided with a plurality of task executors, wherein each task executor is configured to execute one joint learning task to be executed.

[0018] According to an embodiment of the present application, a number of the plurality of task executors to be started is determined according to a current resource idle condition.

[0019] According to an embodiment of the present application, the cloud service component further includes:

[0020] A registration unit is configured to register resources according to resource information of a user to obtain registered resources.

[0021] According to an embodiment of the present application, the cloud service component further includes:

[0022] A storage unit is configured to store the at least one joint learning task.

[0023] According to an embodiment of the present application, the joint learning task includes at least one of a task identifier, a task name, a task application scenario, a task required algorithm, a task joint mode, and a task creator.

[0024] According to an embodiment of the present application, the cloud service component further includes:

[0025] A cache unit is configured to cache the joint learning task to be executed to a message queue.

[0026] According to an embodiment of the present application, the client component further includes:

[0027] A deployment unit is configured to deploy a local environment and initial data.

[0028] A processing unit is configured to perform data cleaning and / or data heterogeneous processing on the initial data to obtain training data.

[0029] The feature engineering unit is configured to process the to-be-trained data to obtain the target data.

[0030] According to an embodiment of the present application, the multi-task parallel joint learning engine architecture system described above further comprises:

[0031] The generation component is configured to generate a global model based on the training results of the client component and the cloud service component.

[0032] The storage component is configured to store the global model to a model library.

[0033] According to the multi-task parallel joint learning engine architecture system of the embodiment of the present application, the client component comprises a training unit and a plurality of clients, which are respectively configured to obtain local training results based on preset local model training target data and upload the local training results to the cloud service component; the cloud service component comprises: a synchronization service unit configured to receive and synchronize the local training results to a task control unit; a task processing library configured to store a plurality of joint learning tasks to obtain a task list; a task consumption unit configured to determine at least one to-be-executed joint learning task from the task list; and a task control unit configured to execute the at least one to-be-executed joint learning task, and match a best engine function module according to an execution result and the local training result to aggregate the local training result, thereby updating the task list. Thus, the problem of queuing and waiting due to too many tasks created by a user is solved, a general processing module is abstracted through a micro-service architecture to reduce code redundancy and repeated development, thereby improving the processing efficiency of the learning engine, and the multi-task parallel joint learning task is realized.

[0034] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0035] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, including the accompanying drawings, wherein:

[0036] Figure 1 FIG. 1 is a block schematic diagram of a multi-task parallel joint learning engine architecture system according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] Embodiments of the present application are described in detail below with reference to the accompanying drawings. Examples of the embodiments are shown in the drawings, in which the same or similar components are denoted by the same or similar reference numerals throughout. The embodiments described below are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0038] A multi-task parallel joint learning engine architecture system according to an embodiment of the present application is described below with reference to the accompanying drawings. In order to solve the problem that the user needs to queue and wait due to too many created tasks, the system cannot better mobilize and manage multiple joint participants, and thus the joint learning efficiency is reduced, the present application provides a multi-task parallel joint learning engine architecture system, which includes a client component and a cloud service component. The client component includes a training unit and multiple clients, which are respectively used to obtain a local training result based on preset local model training target data and upload the local training result to the cloud service component. The cloud service component includes a synchronization service unit, a task processing library, a task consumption unit, and a task control unit. The synchronization service unit is used to receive and synchronize the local training result to the task control unit. The task processing library is used to store multiple joint learning tasks to obtain a task list. The task consumption unit is used to determine at least one to-be-executed joint learning task from the task list. The task control unit is used to execute the at least one to-be-executed joint learning task, match a best engine function module according to an execution result and the local training result, aggregate the local training result through the best engine function module, and update the task list according to an aggregation processing result. Thus, the problem that the user needs to queue and wait due to too many created tasks is solved. A general processing module is abstracted through a micro-service architecture to reduce code redundancy and repeated development, thereby improving the processing efficiency of the learning engine to implement multi-task parallel joint learning tasks.

[0039] Specifically, Figure 1 A block diagram of a multi-task parallel joint learning engine architecture system according to an embodiment of the present application is provided.

[0040] As Figure 1 shown, the multi-task parallel joint learning engine architecture system 10 includes a client component 100 and a cloud service component 200.

[0041] The client component 100 includes a training unit 101 and multiple clients 102. The training unit 101 is used to obtain a local training result based on preset local model training target data. The multiple clients 102 are used to upload the local training result to the cloud service component. The cloud service component 200 includes a synchronization service unit 201, a task processing library 202, a task consumption unit 203, and a task control unit 204. The synchronization service unit 201 is used to receive and synchronize the local training result to the task control unit. The task processing library 202 is used to store multiple joint learning tasks created based on registered resources to obtain a task list. The task consumption unit 203 is used to determine at least one to-be-executed joint learning task from the task list. The task control unit 204 is used to execute the at least one to-be-executed joint learning task, match a best engine function module according to an execution result and the local training result, aggregate the local training result through the best engine function module, and update the task list according to an aggregation processing result.

[0042] According to one embodiment of the present application, the cloud service component 200 further comprises: a registration unit configured to register resources according to resource information of a user to obtain registered resources; and a storage unit configured to store at least one joint learning task.

[0043] Specifically, in the embodiments of the present application, when a user creates a multi-task according to his own needs, the user first registers on the task platform according to the registered resource information through the registration unit in the cloud service component 200 to obtain registered resources; secondly, after registering the resource information, the user can create at least one joint learning task through the task platform front-end page or directly join a suitable learning task; finally, the created at least one joint learning task or the joined learning task is stored in the task processing library 202 through the storage unit to obtain a task list, wherein the stored joint learning task information includes at least one of a task identifier, a task name, a task application scenario, a task algorithm, a task joint mode (horizontal / vertical), and a task creator. The specific task information can be created according to the user's own needs, which is not limited here.

[0044] Further, after the creation of at least one joint learning task is completed, the present application embodiment needs to determine at least one to-be-executed joint learning task from the task list through the task consumption unit 203 and consume and execute the at least one joint learning task.

[0045] It should be noted that since there can be multiple to-be-executed learning tasks at the same time in the embodiments of the present application, first, at least one joint learning task in the task list is placed in the message queue for caching through the caching unit, and then the task control unit 204 determines the specific number of task executors in the task control unit 204 to be started according to the current resource idle condition, wherein each task executor is used to execute at least one to-be-executed joint learning task to obtain an execution result, and in each execution of a learning task, a best engine function module is matched according to a priority order from a Lambda store according to the execution result and a local training result obtained by the training unit 101 based on a preset local model training target data, and the local training result is aggregated and processed through the best engine function module, and then the task list is updated according to the aggregation processing result. It should be noted that after obtaining the local training result, the task executor needs to interact with the client component 100 through the synchronization service unit 201, synchronize and upload the local training information of the client component 100 to the task control unit 204 for aggregation and other operations.

[0046] The engine function modules include at least one of an aggregation strategy, a heterogeneous strategy optimization, a joint knowledge distillation, an adaptive mechanism, and an exception handling strategy. Meanwhile, Lambda management manages the priorities of the engine function modules, and the engine function modules are relatively independent and dependent on each other. The appropriate module is selected for execution according to the scene requirement of a user, so that code redundancy and repeated development can be reduced, and the engine processing efficiency can be improved.

[0047] According to an embodiment of the present application, the client component 100 further comprises a deployment unit configured to deploy a local environment and initial data; a processing unit configured to perform data cleaning and / or data heterogeneous processing on the initial data to obtain training data; and a feature engineering unit configured to process the training data to obtain target data.

[0048] Specifically, in the embodiment of the present application, when the client component 100 performs task training, first, the local environment and the initial data need to be deployed by the deployment unit, and the initial data is preprocessed by the processing unit, including data cleaning and / or data heterogeneous processing on the initial data to obtain training data; second, the feature engineering unit performs local model training on the training data to obtain target data, wherein during the training process, the learning task heartbeat can be monitored in real time, and the learning task state is updated to the task list. When the learning task is abnormal, the processing mechanism in the exception handling module is executed to form a new task state; finally, the local training result is uploaded to the cloud service component 200, and the new task state is updated to the task list in the task processing library 202 to obtain the training result of the client component 100. The environment deployment, initial data preprocessing, and training process of the training data in the embodiment of the present application can adopt the processing method in the related art, which will not be discussed in detail here.

[0049] It should be noted that the learning task of the client component 100 in the embodiment of the present application is too much, so at least one joint learning task in the task list also needs to be placed in the message queue for caching and queuing, and then uploaded to the task control unit 204 for aggregation and other operations.

[0050] According to an embodiment of the present application, the multi-task parallel joint learning engine architecture system 10 further comprises a generation component configured to generate a global model based on the training results of the client component 100 and the cloud service component 200; and a storage component configured to store the global model to a model library.

[0051] Specifically, the embodiments of the present application obtain training results of joint learning tasks according to the client component 100 and the cloud service component 200 respectively, and generate a global model based on the training results, and store the global model to a model library through the storage component, so as to execute corresponding models according to the scene requirements of users.

[0052] According to the multi-task parallel joint learning engine architecture system, the client component includes a training unit and a plurality of clients, which are respectively used to obtain local training results based on preset local model training target data and upload the local training results to the cloud service component; the cloud service component includes a synchronization service unit, a task processing library, a task consumption unit and a task control unit, the synchronization service unit is used to receive and synchronize the local training results to the task control unit; the task processing library is used to store a plurality of joint learning tasks to obtain a task list; the task consumption unit is used to determine at least one to-be-executed joint learning task from the task list; the task control unit is used to execute the at least one to-be-executed joint learning task, and match a best engine function module according to an execution result and the local training result to aggregate the local training result, so as to update the task list. Thus, the problem that a user needs to queue and wait due to too many created tasks is solved, a general processing module is abstracted through the micro-service architecture, so as to reduce code redundancy and repeated development, thereby improving the processing efficiency of the learning engine, and realizing the multi-task parallel joint learning task.

[0053] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0054] In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0055] Any processes or methods described in the flowcharts or otherwise described herein can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for implementing specific logic functions (or steps) of the application, and alternate implementations are possible. The various steps or functions described in the flowcharts or otherwise described herein can be implemented as program instructions (i.e., as one or more modules of computer program code) in any of various forms, including but not limited to program components, applets, threads of execution, procedures, functions, etc., whether implemented in hardware, software, firmware, or their combination. It will be understood that the various steps or functions described in the flowcharts or otherwise described herein can be implemented by any number of hardware devices or software modules, including but not limited to processors, hardwired circuitry, software modules, firmware modules, etc. Thus, the steps or functions of the flowcharts or other descriptions herein can be embodied in any of a wide variety of forms, including but not limited to software code, hardware logic, software code and hardware logic, firmware code, etc. The various forms of the application can be implemented in any of a variety of ways, including as software code, firmware code, hardware logic, etc. The software code, firmware code, and / or hardware logic can be stored, for example, on a computer-readable medium, which can be any medium readable by a computer or other instruction execution system, including but not limited to any of various types of memory devices, including volatile memory, non-volatile memory, etc. The software code, firmware code, and / or hardware logic can be executed by an instruction execution system, which can be any system that can fetch, decode, and execute instructions, including but not limited to a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from a computer-readable medium.

[0056] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a list of executable instructions for implementing the logic function, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions. For purposes of this specification, a "computer-readable medium" can be any medium that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be any one of many physical media, including but not limited to electronic, magnetic, optical, electromagnetic, infrared, and semiconductor systems (or apparatuses or devices). A non-exhaustive list of more specific computer-readable medium examples (inclusion of which is not a limitation of the application) includes the following: a portable computer diskette (magnetic), a hard disk (magnetic), a RAM (random access memory, magnetic), a ROM (read only memory, magnetic), an EPROM (erasable programmable read only memory, magnetic), a FLASH memory (magnetic), an optical fiber (optical), and a portable compact disc read only memory (CDROM, magnetic). Additionally, the computer-readable medium can be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

[0057] It will be understood that various parts of the present application can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As such, if implemented in hardware, the various steps or methods can be implemented in any of a variety of ways, including but not limited to as discrete logic circuitry having logic gates for implementing logic functions on data signals, as application specific integrated circuits having appropriate combinational logic gates, as programmable logic arrays (PLAs), as field programmable gate arrays (FPGAs), etc.

[0058] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment method can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0059] In addition, each functional unit in each embodiment of the present application can be integrated into one processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The integrated module can be realized in the form of hardware or in the form of a software functional module. The integrated module, if realized in the form of a software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium.

[0060] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present application.

Claims

1. A multi-task parallel federated learning engine architecture system, characterized in that, The application relates to a federated learning method and system. The client component and the cloud service component, The client component comprises: a training unit configured to train target data based on a preset local model training target data to obtain a local training result; a plurality of clients configured to upload the local training result to the cloud service component; The cloud service component comprises: a synchronization service unit configured to receive and synchronize the local training result to a task control unit; a task processing library configured to store a plurality of federated learning tasks created based on registered resources to obtain a task list; a task consumption unit configured to determine at least one to-be-executed federated learning task from the task list; The task control unit is configured to execute the at least one to-be-executed federated learning task, match a best engine function module according to an execution result and the local training result in a Lambda store in a priority order, perform aggregation processing on the local training result through the best engine function module, and update the task list according to an aggregation processing result. A cache unit is configured to cache the to-be-executed federated learning task to a message queue. The task control unit is provided with a plurality of task executors, wherein each task executor is configured to execute at least one to-be-executed federated learning task to obtain an execution result; and the number of the plurality of task executors to be started is determined according to a current resource idle condition. The engine function module comprises at least one of an aggregation strategy, a heterogeneous strategy optimization, a federated knowledge distillation, an adaptive mechanism and an exception handling strategy; the priority of the engine function module is managed by a Lambda management, and a suitable engine function module is selected according to a user's scene requirement to perform execution.

2. The multi-task parallel federated learning engine architecture system of claim 1, wherein, The cloud service component further comprises: a registration unit configured to register resources according to user's resource information to obtain registered resources.

3. The multi-task parallel federated learning engine architecture system of claim 1, wherein, The cloud service component further comprises: a storage unit configured to store the at least one federated learning task.

4. The multi-task parallel federated learning engine architecture system of claim 3, wherein, The federated learning task comprises at least one of a task identifier, a task name, a task application scene, a task required algorithm, a task federated mode and a task creator.

5. The multi-task parallel federated learning engine architecture system of claim 1, wherein, The client component further comprises: a deployment unit configured to deploy a local environment and initial data; a processing unit configured to perform data cleaning and / or data heterogeneous processing on the initial data to obtain to-be-trained data; a feature engineering unit configured to process the to-be-trained data to obtain the target data.

6. The multi-task parallel federated learning engine architecture system of claim 1, wherein, The application further comprises: a generation component configured to generate a global model based on training results of the client component and the cloud service component; a storage component configured to store the global model to a model library.

Images