Edge-Facing Federated Learning Deployment System Architecture, Method, Device, and Storage Medium

Abstract

The present invention discloses an edge federated learning deployment architecture, method, device, and storage medium, relating to the field of computer technology. The edge federated learning deployment architecture includes an edge terminal layer, a global center layer, and a module interface layer. The module interface layer is used to provide Restful-style API interfaces for data interaction between edge devices and central servers. The global center layer is used to configure federated learning task codes according to the edge device information transmitted by the edge terminal layer, encapsulate the federated learning task codes using container technology to obtain image containers, and the edge terminal layer is used to obtain and run the image containers to run federated learning tasks in the operating environment constructed by the federated learning task codes. This application can reduce the computing power requirements for edge devices and improve the simplicity and applicability of federated learning task deployment.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular, to an edge federated learning deployment system architecture, method, device, and storage medium. Background Art

[0002] With the advent of the big data era, the computing power of computers and the network bandwidth have been greatly improved, and machine learning has also been applied more and more widely, becoming the key to many modern technologies today. Nowadays, many research teams are constantly exploring the possibility of transferring machine learning to the network edge, putting forward new concepts such as cloud intelligence and edge intelligence, and deriving application scenarios such as smart cities and intelligent transportation. However, with the in-depth research and exploration, many problems have also been exposed.

[0003] Problem 1: Traditional machine learning methods require a large amount of high-quality data to ensure the accuracy of the model, so they require a large model training cost. Edge devices are often resource-constrained in terms of computing power, network bandwidth, etc., which poses strict conditions for the deployment and application of traditional machine learning methods on edge devices.

[0004] Problem 2: The protection of data privacy and information security is gradually becoming a worldwide trend. A series of regulations and regulations implemented have imposed restrictions on the processes of data collection, transmission, storage, processing, etc. The behavior of directly collecting and using others' data in traditional deep learning frameworks is no longer allowed, increasing the difficulty of related research.

[0005] Problem 3: In the era of big data, the concept of "data is an asset" is increasingly accepted by enterprises. Enterprises pay more and more attention to local data within the enterprise and have less and less data exchange with other enterprises. The phenomena of "data barriers" and "data islands" are becoming more and more serious, and it is increasingly difficult to ensure the data quality required by traditional machine learning methods. Traditional machine learning methods are greatly restricted.

[0006] By combining the federated learning framework with edge computing to construct an edge federated learning framework for local training of edge device data and global optimization of model parameters, the service quality of the local model can be improved without increasing the number of local data on edge devices, and it can better adapt to resource-constrained edge devices. At the same time, the edge federated learning framework can effectively meet the user's data security needs and the need for service multi-source heterogeneous data fusion, and is one of the effective solutions for edge computing to ensure data security in the big data era.

[0007] Despite the many advantages of the edge federated learning framework, there are still the following problems: when the edge federated learning system is actually implemented, all the participating edge devices often cannot achieve unified machine configuration and network environment. Since the traditional application deployment method is to install applications through plugins or scripts, the operation, configuration, management, and entire lifecycle of the application will be bound to the operating system of the current edge device, which will result in relatively high upfront training costs and human costs for training environment preparation, that is, there is a problem of difficult deployment and configuration of edge devices. In this regard, although certain functions can be achieved by creating virtual machines, the granularity of virtual machines themselves is very large, which is not conducive to transplantation and deployment, and there is also a problem of relatively high labor costs.

[0008] Currently, the commonly used K8S cluster is used for containerized application management of edge devices to simplify the deployment process. The full name of K8S is Kubernetes, which is an open-source container orchestration and management tool for portable containers, dedicated to solving the problem of managing Docker containerized applications on multiple hosts in the above cloud platform. The goal of K8S is to make the deployment of containerized applications simple and efficient, and it provides a mechanism for application deployment, planning, updating, and maintenance.

[0009] When facing the problem of deployment and configuration of resource-constrained edge devices in the edge federated learning framework, although container technology is more friendly compared to traditional application deployment methods and virtual machine deployment methods, when facing application scenarios such as federated learning that require frequent replacement of configurations inside containers, the problem of container configuration management is prominent. The container orchestration and management tool K8S in related technologies also has problems that are not friendly to edge devices, specifically manifested as follows:

[0010] 1) The deployment of the K8S cluster environment itself is complicated. When facing the problem of deploying edge devices with a large number of edge devices and inconsistent environment configurations, it will also generate relatively high upfront training costs and human costs for training environment preparation;

[0011] 2) The K8S cluster environment is mainly oriented to large-scale applications, and most edge devices do not support accessing the K8S cluster due to system problems or configuration problems;

[0012] 3) There are certain security risks in the interface interaction of the K8S container management platform itself. If the configuration is improper when accessing devices, there is a certain possibility of causing the entire K8S cluster to crash, and the large number of edge devices and inconsistent environment configurations will greatly increase the risk of the K8S cluster crashing. Summary of the Invention

[0013] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes an edge-oriented federated learning deployment system architecture, method, device and storage medium, which can improve the simplicity and applicability of federated learning task deployment.

[0014] On the one hand, an embodiment of the present invention provides an edge-oriented federated learning deployment system architecture, including an edge terminal layer, a global center layer and a module interface layer;

[0015] The edge terminal layer is deployed in edge devices of the edge federated learning system, and the global center layer is deployed in a central server of the edge federated learning system;

[0016] The module interface layer is used to provide API interfaces for data interaction between the edge devices and the central server;

[0017] The global center layer is used to configure federated learning task codes according to the edge device information transmitted by the edge terminal layer, and encapsulate the federated learning task codes using container technology to obtain an image container;

[0018] The edge terminal layer is used to obtain and run the image container to run the federated learning task in the running environment constructed by the federated learning task code.

[0019] According to some embodiments of the present invention, the module interface layer includes a Restful-style device information upload API interface and an image container and file download API interface.

[0020] According to some embodiments of the present invention, the global center layer includes a task configuration module and a model optimization module;

[0021] The task configuration module is used to configure federated learning task codes according to the edge device information transmitted by the edge terminal layer, and encapsulate the federated learning task codes using container technology to obtain an image container;

[0022] The model optimization module is used to optimize the model parameters of the models trained by each edge device in the edge terminal layer, and aggregate and update the model parameters.

[0023] According to some embodiments of the present invention, the global center layer further includes an algorithm library and an image container library;

[0024] The algorithm library is used to provide the federated learning task codes for the task configuration module;

[0025] The image container library is used to store the image containers encapsulated by the task configuration module.

[0026] According to some embodiments of the present invention, the edge terminal layer includes a Linux system building module, an image container acquisition module, and a federated learning task running module;

[0027] The Linux system building module is used to transplant the Bash command processor to the edge device through a terminal emulator, and pull a Linux image through the Bash command processor to install the Linux system on the edge device;

[0028] The image container acquisition module is used to obtain and run an image container from the global center layer through the image container and the file download API interface in the root environment of the Linux system to build a running environment;

[0029] The federated learning task running module is used to obtain the shell script of the federated learning task from the global center layer through the image container and the file download API interface in the root environment of the Linux system, and run the shell script in the running environment.

[0030] According to some embodiments of the present invention, the container technology is Docker container technology, and the image container is a Docker image container.

[0031] According to some embodiments of the present invention, the task configuration module includes a configuration unit, a code download unit, a compilation unit, and an image generation unit;

[0032] The configuration unit is used to obtain edge device information through the device information upload API interface, and determine the configuration information of the federated learning task according to the edge device information, where the configuration information includes a code download path, a code compilation instruction, and a Dockerfile;

[0033] The code download unit is used to download the federated learning task code from the algorithm library according to the code download path in the Docker container;

[0034] The compilation unit is used to execute the code compilation instruction to compile the federated learning task code and generate a compilation result file;

[0035] The image generation unit is used to generate a Docker image container according to the Dockerfile and the compilation result file.

[0036] On the other hand, an embodiment of the present invention also provides a method for deploying edge federated learning. The method for deploying edge federated learning is applied to the central server of the edge federated learning system. The edge federated learning system is deployed with the architecture of the edge federated learning deployment system described in the previous embodiments. The method for deploying edge federated learning includes the following steps:

[0037] Obtain edge device information of edge devices through the module interface layer;

[0038] Configure the federated learning task code according to the edge device information;

[0039] Using container technology to encapsulate the federated learning task code to obtain a mirror container;

[0040] In response to the operation instruction, the image container is sent to the edge device through the module interface layer.

[0041] On the other hand, an embodiment of the present invention further provides an edge federated learning deployment device, including:

[0042] at least one processor;

[0043] at least one memory for storing at least one program;

[0044] When the at least one program is executed by the at least one processor, the at least one processor implements the edge-oriented federated learning deployment method as described above.

[0045] On the other hand, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the edge-oriented federated learning deployment method as described above.

[0046] The above technical solution of the present invention has at least one of the following advantages or beneficial effects: the edge-oriented federated learning deployment system architecture of the present application includes an edge terminal layer, a global center layer and a module interface layer, the module interface layer is used to provide an API interface for data interaction between edge devices and central servers, the global center layer is used to configure the federated learning task code according to the edge device information transmitted by the edge terminal layer, and the container technology is used to encapsulate the federated learning task code to obtain a mirror container, and the edge terminal layer is used to obtain and run the mirror container to run the federated learning task in the operating environment constructed by the federated learning task code. The edge-oriented federated learning deployment system architecture is deployed in the edge federated learning system, and the central server can complete the operating environment configuration work of the edge device federated learning related tasks, and delegate the container configuration information through the API interface provided by the module interface layer. The edge device only needs to complete the acquisition and installation configuration of the mirror container through the module interface layer to access the federated learning system, which reduces the computing power requirements for the edge device and improves the simplicity and applicability of the deployment of federated learning tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the edge federated learning deployment system architecture provided by an embodiment of the present invention;

[0048] Figure 2 It is a process flowchart for an Android operating system terminal provided by an embodiment of the present invention to implement a federated learning task;

[0049] Figure 3 It is a process flowchart for a central server provided by an embodiment of the present invention to implement the deployment process of a federated learning task;

[0050] Figure 4 It is a flowchart of a method for edge federated learning deployment provided by an embodiment of the present invention;

[0051] Figure 5 It is a schematic diagram of a device for edge federated learning deployment provided by an embodiment of the present invention. Detailed implementation manners

[0052] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation of the present invention.

[0053] In the description of the present invention, it should be understood that for the orientation description, such as the orientation or positional relationship indicated by up, down, left, right, etc., is based on the orientation or positional relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0054] In the description of the present invention, if the first, second, etc. are described, it is only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features or implicitly indicating the sequence relationship of the indicated technical features.

[0055] Before further elaborating on the embodiments of the present application, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are applicable to the following explanations.

[0056] Federated Learning: Federated Learning is an emerging basic technology for artificial intelligence. Its design goal is to carry out efficient machine learning among multiple parties or computing nodes while ensuring information security during big data exchange, protecting terminal data and personal data privacy, and ensuring legality and compliance. Among them, the machine learning algorithms that can be used in federated learning are not limited to neural networks, but also include important algorithms such as random forests. Federated learning is expected to become the foundation of the next generation of artificial intelligence collaborative algorithms and collaborative networks.

[0057] Data silos: Data is stored and maintained independently in different departments, isolated from each other, forming physical silos. Intuitively speaking, the data between each other cannot be coordinated or communicated, and are in a state of separation.

[0058] Federated learning system framework: After users have trained locally, they upload models that are not related to user information. The cloud server can integrate model information from different users to obtain a better global model, and then send it back to the user for further training. Repeating this process, an excellent model that can be applied in reality can be obtained, while ensuring that user privacy information is not leaked.

[0059] Multi-source heterogeneity: In simple terms, it means that a whole is composed of components from multiple different sources, including mixed data (including structured and unstructured) and discrete data (data distributed in different systems or platforms). The Internet is a typical heterogeneous network. Faced with limited user personality data, ordinary machine learning algorithms cannot effectively process multi-source heterogeneous data, resulting in models that are only applicable to a certain group of people, with poor generalization and inability to serve a wider range of people.

[0060] Microservice: Microservice is an architectural pattern that advocates dividing a single application into a group of small services. Each service runs in its own independent process, and services collaborate with each other using lightweight communication mechanisms. Each service is built around a specific business and can be fully automated and independently deployed. There is a minimum centralized management of these services, and they can be developed in different development languages ​​and use different data storage technologies.

[0061] Shell script: Shell script is similar to batch processing under Windows / Dos, that is, various commands are pre-placed in a file, which is a program file for one-time execution. It is mainly used by administrators for setting or management. Shell script is more powerful than batch processing under Windows and more efficient than programs edited with other programming programs. Shell script uses commands under Linux / Unix.

[0062] Docker: Docker is an open-source application container engine that allows developers to package their applications and dependent packages into a portable container and then deploy it to any popular Linux machine. It can also achieve virtualization. Containers use a sandbox mechanism entirely and have no interfaces with each other.

[0063] Docker image: It is a read-only file package generated according to the format specified by Docker. An application can be encapsulated in this Docker image and distributed to different hosts. Software containers can be started through the Docker image to run services.

[0064] Docker registry: It is a centralized repository for storing Docker images. Different hosts can download Docker images from this repository.

[0065] Introduction to Termux software: Termux is an Android terminal emulator and an application that provides a Linux environment. It does not require rooting the device or any settings and is ready to use out of the box, automatically installing a basic Linux system.

[0066] Restful API: It is an API in the REST (Representational State Transfer) style, that is, an application programming interface, which is a set of definitions, programs, and protocols. Through the API interface, computer software can communicate with each other. A main function of the API is to provide a general function set. Programmers can develop applications by calling API functions, which can reduce programming tasks. The API is also a kind of middleware that provides data sharing for various different platforms. RESTful API requires the front end to send requests in a predefined syntax format, and the server only needs to define a unified response interface without having to parse various requests.

[0067] The embodiment of the present invention provides an architecture for an edge federated learning deployment system, referring to Figure 1 , including an edge terminal layer, a global center layer, and a module interface layer. The edge terminal layer is deployed in the edge devices of the edge federated learning system, and the global center layer is deployed in the central server of the edge federated learning system. The module interface layer is used to provide corresponding API interfaces for various types of data interactions between the edge devices and the central server. The global center layer is used to configure the federated learning task code according to the edge device information transmitted by the edge terminal layer, and encapsulate the federated learning task code using container technology to obtain an image container. The edge terminal layer is used to obtain and run the image container to run the federated learning task in the running environment constructed by the federated learning task code.

[0068] In some embodiments, the container technology may be Docker container technology. Correspondingly, the image container is a Docker image container.

[0069] Specifically, the edge terminal layer is deployed on edge devices and serves as a participating module of the edge devices in the edge federated learning system. It receives federated learning-related requests from edge devices (such as computers, mobile phones, etc.) and provides relevant application services in the edge terminal layer according to the requests. Since the resources of edge devices are limited, in the embodiments of the present invention, Docker images are obtained from the central server through the API interfaces corresponding to the module interface layer to provide a running environment for the federated learning service in the edge devices. The functions implemented by the edge terminal layer in the edge devices include but are not limited to local model training, model parameter uploading and updating, and local model optimization and updating.

[0070] The global central layer is deployed on the central server and serves as a participating module of the central server in the edge federated learning system. It interacts with the edge terminal layer through the module interface layer, configures the federated learning task code according to the edge device information, encapsulates the federated learning task code using container technology to obtain an image container, and deploys the image container to the edge devices. The functions implemented by the global central layer include but are not limited to global optimization of model parameters, aggregated update of model parameters, and task deployment.

[0071] The module interface layer is deployed between the edge devices and the central server and provides RESTful-style API interfaces as the communication means between the edge devices and the central server in the edge federated learning system. It includes services for information communication between the edge devices and the central server, corresponding to the services of the edge terminal layer and the global central layer. The module interface layer includes but is not limited to API interfaces for services such as local model uploading API interface, global model deployment API interface, global model update API interface, Docker image and file deployment API interface.

[0072] The deployment system architecture of the edge federated learning of the embodiments of the present invention is a microservices architecture. Using the microservices architecture effectively splits the application, minimizes the applications contained in the edge devices, and runs independently in operating systems such as Android inside the edge devices without external dependencies, effectively reducing the coupling degree of the edge devices in the edge federated learning system. The communication between the central server and the edge device services is implemented through RESTful-style API interfaces. The global central layer completes the configuration work of the running environment for federated learning-related tasks, while the edge terminal layer pulls the required Docker containers containing the running environment from the global central layer through the API interfaces provided by the module interface layer, and completes the relevant tasks involved in the edge devices of the edge federated learning in the Docker containers, thereby completing the simple deployment of the edge federated learning system.

[0073] According to some specific embodiments of the present invention, the global center layer includes a task configuration module and a model optimization module.

[0074] The task configuration module is used to configure the federated learning task code according to the edge device information transmitted by the edge terminal layer, and encapsulate the federated learning task code using container technology to obtain an image container. In this embodiment, when it is determined according to the edge device information that the edge device is a resource-constrained device, the central server selects the federated learning task code corresponding to the relevant algorithm under the lightweight machine learning framework to complete the federated learning-related tasks; when it is determined according to the edge device information that the edge device is not a resource-constrained device, the federated learning task code corresponding to the machine learning-related algorithm with higher resource requirements should be selected to provide better services. The edge device information includes, but is not limited to, device configuration information such as CPU processing speed, memory capacity, and hard disk capacity. The central server performs a weighted calculation on multiple device configuration information according to the values of each device configuration information and the corresponding weights to obtain a device resource score, and determines the federated learning task according to the mapping relationship between the device resource score stored in the central server and the machine learning algorithm. After determining the federated learning task, the federated learning task code is encapsulated using Docker container technology to obtain a Docker image container.

[0075] The model optimization module is used to obtain the model parameters through the local model upload API interface, optimize the model parameters of the models trained by each edge device in the edge terminal layer, and perform aggregation and update on the model parameters.

[0076] According to some specific embodiments of the present invention, the global center layer further includes an algorithm library and an image container library. The algorithm library stores various machine learning algorithm codes, and the task configuration module can obtain the corresponding federated learning task code from the algorithm library according to the edge device information. The image container library is used to store the encapsulated image containers, and the edge device can obtain the corresponding image containers by accessing the image container library through the API interface.

[0077] According to some specific embodiments of the present invention, the edge terminal layer includes a Linux system building module, an image container obtaining module, and a federated learning task running module.

[0078] The Linux system building module is used to transplant the Bash command processor to the edge device through the terminal emulator, and pull the Linux image through the Bash command processor to install the Linux system on the edge device.

[0079] The image container obtaining module is used to obtain and run the image container from the global center layer through the image container and file download API interface in the root environment of the Linux system to build a running environment.

[0080] The federated learning task running module is used to obtain the shell script of the federated learning task from the global central layer through the mirror container and the file deployment API interface in the root environment of the Linux system, and run the shell script in the running environment.

[0081] Specifically, the edge terminal layer is deployed on the Android operating system terminal and runs in the complete Linux environment built under it. Microservices are developed using Docker container technology, which provides all functions for compiling, uploading, downloading, starting, and stopping Docker containers. At the same time, there are interfaces in the Docker container for obtaining Android local data for local model training and providing services for relevant federated learning applications. The services provided by the edge terminal layer do not require local participation, and only need to upload and download the container. The rest of the environment configuration is located in the global central layer.

[0082] Refer to Figure 2 , when deploying the edge terminal layer on the Android operating system terminal, the process of the Android operating system terminal implementing the federated learning task is as follows:

[0083] S11, use Termux to open a new Bash command processor on Android to listen to the input stream from the mobile phone keyboard for scheduling;

[0084] S12, use the Bash command processor to pull the complete Linux image and install it locally in the Android system as another environment;

[0085] S13, enter the root environment of the Linux system, obtain the administrator permission and modify the configuration file, pull the Docker image from the mirror container library of the central server to run locally, and build a running environment according to the specified federated learning task code;

[0086] S14, in the root environment of the Linux system, through the mirror container and the file deployment API interface provided by the module interface layer, obtain the shell script (.sh file) from the central server, and run the shell script (.sh file) in the built running environment;

[0087] S15, after running the shell script, enter the continuous working state, and automatically recycle the Docker image after the federated learning task is completed.

[0088] According to some specific embodiments of the present invention, the task configuration module includes a configuration unit, a code download unit, a compilation unit, and an image generation unit.

[0089] The configuration unit is used to obtain edge device information through the device information upload API interface, and determine the configuration information of the federated learning task according to the edge device information. Among them, the configuration information includes the code download path, code compilation instructions, and Dockerfile file;

[0090] The code download unit is used to download the federated learning task code from the algorithm library according to the code download path within the Docker container.

[0091] The compilation unit is used to execute the code compilation instructions to compile the federated learning task code and generate a compilation result file.

[0092] The image generation unit is used to generate a Docker image container according to the Dockerfile file and the compilation result file.

[0093] Specifically, the global center layer can be deployed in the central server of common server operating systems such as Windows server and CentOS. The central server includes an image container library, and there are multiple Docker images stored in the image container library. The training environment of the federated learning task is configured in the Docker image. There is an interaction interface between the edge terminal layer and the global center layer in the module interface layer, which can provide services for the edge terminal layer. The services provided include but are not limited to federated learning environment deployment, global model update, etc.

[0094] Refer to Figure 3 , the process of the central server implementing the federated learning task deployment is as follows:

[0095] S21, obtain the edge device information of the edge device through the API interface provided by the module interface layer, and determine the federated learning task code under the appropriate machine learning framework according to the edge device information. In this implementation, the appropriate federated learning task code is selected according to the device characteristics of the edge device, with high flexibility and strong adaptability.

[0096] S22, run the Docker container, and determine the corresponding code download path, code compilation instructions, and Dockerfile file. The code download path refers to the determined address of the federated learning task code; the code compilation instructions are used to compile the federated learning task code to generate a compilation result file; the Dockerfile file is used for the generation of the Docker image.

[0097] S23, run the relevant commands, and download the corresponding federated learning task code from the algorithm library within the Docker container according to the code download address, and save it in the Docker container, so that the federated learning-related tasks are encapsulated in the Docker container.

[0098] S24, run relevant commands, execute code compilation instructions to compile the federated learning task code and generate a compilation result file, and store the generated compilation result file in the Docker container.

[0099] S25, run relevant commands to generate a Docker image according to the Dockerfile file and the compilation result file. After the Docker container obtains the above compilation result file, it starts the Dockerflie file to generate the required Docker image according to the configuration information generated by the startup parameters such as the service name, port, and network mode.

[0100] S26, determine the image container information from the startup parameters, and store the Docker image in the local image container library of the global central layer server according to the image warehouse information.

[0101] According to some specific embodiments of the present invention, the edge-oriented federated learning deployment system architecture built based on the microservice architecture has the following advantages compared with the prior art:

[0102] 1) For edge federated learning, more attention is paid to resource-constrained edge devices. The K8S management tool commonly used in dealing with edge federated learning deployment issues has high performance requirements for devices. Most edge devices in actual application scenarios do not support K8S deployment, and cannot be smoothly carried out when deploying federated learning for edge devices. The edge federated learning deployment system architecture of the embodiment of the present invention starts from the architectural design, and transmits the container configuration information through the network layer RESTful API interface supported by all edge devices. The federated learning system can also be deployed for resource-constrained devices.

[0103] 2) Easier edge device deployment. Compared with the virtual machine deployment method, the embodiment of the present invention uses the smaller-granularity Docker container technology to complete the deployment. The edge device only needs to complete the installation and configuration of the local Docker container to access the federated learning system, which is easier to configure.

[0104] 3) Easier and safer deployment. The embodiment of the present invention adopts a modular design approach. In the microservice architecture, it is only necessary to add the required functions to a specific service without affecting the architecture of the overall process. The server not only participates in federated learning-related tasks, but also completes the operating environment configuration work of the edge device federated learning-related tasks, and delegates the container configuration information through the network layer interface. The network layer interface is more secure than other cluster interfaces and will not affect the overall system due to a single configuration.

[0105] On the other hand, the embodiments of the present invention further provide a method for deploying edge federated learning. The method for deploying edge federated learning is applied to the central server of the edge federated learning system. The edge federated learning system is deployed with the architecture of the system for deploying edge federated learning described in the foregoing embodiments. Referring to Figure 4 , the method for deploying edge federated learning includes but is not limited to steps S110, S120, S130, and S140:

[0106] Step S110, obtaining edge device information of edge devices through the module interface layer;

[0107] Step S120, configuring federated learning task codes according to the edge device information;

[0108] Step S130, encapsulating the federated learning task codes using container technology to obtain an image container;

[0109] Step S140, in response to an operation instruction, sending the image container to the edge device through the module interface layer.

[0110] It can be understood that the content in the embodiments of the above-mentioned architecture of the system for deploying edge federated learning regarding the global central layer is applicable to the embodiments of the method for deploying edge federated learning of the present invention. The functions specifically implemented by the method for deploying edge federated learning of the present invention are the same as those of the embodiments of the above-mentioned architecture of the system for deploying edge federated learning, and the beneficial effects achieved are also the same as those of the embodiments of the above-mentioned architecture of the system for deploying edge federated learning.

[0111] Referring to Figure 5 , Figure 5 is a schematic diagram of a device for deploying edge federated learning provided by an embodiment of the present invention. The device for deploying edge federated learning according to the embodiment of the present invention includes one or more control processors and a memory. Figure 5 In

[0112] an example of one control processor and one memory is used. Figure 5 In

[0113] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include memories remotely located relative to the control processor, and these remote memories can be connected to the edge-oriented federated learning deployment device through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0114] Those skilled in the art can understand that Figure 5 the device structure shown in does not constitute a limitation on the edge-oriented federated learning deployment device, and may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0115] The non-transitory software programs and instructions required to implement the edge-oriented federated learning deployment method applied to the edge-oriented federated learning deployment device in the above embodiments are stored in the memory, and when executed by the control processor, implement the edge-oriented federated learning deployment method applied to the edge-oriented federated learning deployment device in the above embodiments.

[0116] In addition, an embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions, which are executed by one or more control processors, enabling the one or more control processors to execute all or some of the steps in the method embodiment of the method for edge federated learning deployment opening. The system can be implemented as software, firmware, hardware, and their appropriate combinations. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory, or other memory technologies, CD-ROM, digital versatile disk (DVD), or other optical disk storage, magnetic cartridges, tapes, magnetic disk storage, or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, a communication medium typically contains computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transmission mechanism, and can include any information delivery medium.

[0117] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made without departing from the spirit of the present invention within the scope of knowledge possessed by those of ordinary skill in the relevant technical field.

Claims

1. An edge-oriented federated learning deployment system architecture, characterized in that, It includes edge terminal layer, global center layer and module interface layer; The edge terminal layer is deployed in the edge device of the edge federated learning system, and the global center layer is deployed in the center server of the edge federated learning system; The module interface layer is used to provide an API interface for data interaction between the edge device and the central server; The global center layer is used to configure the federated learning task code according to the edge device information transmitted by the edge terminal layer, use container technology to encapsulate the federated learning task code to obtain an image container, store the image container in the image container library, and transfer the image container in the image container library to the corresponding edge terminal layer through the module interface layer; the edge device information includes various device configuration information of the device; wherein, when it is determined according to the edge device information that the edge device is a resource-constrained device, the federated learning task code corresponding to the first type of algorithm is selected; when it is determined according to the edge device information that the edge device is not a resource-constrained device, the federated learning task code corresponding to the second type of algorithm is selected; the computing power requirement of the second type of algorithm is higher than that of the first type of algorithm, and the first type of algorithm is an algorithm under the lightweight machine learning framework; The edge terminal layer is used to obtain and run the image container to run the federated learning task in the operating environment constructed by the federated learning task code; Among them, the global center layer is specifically used to perform weighted calculation on multiple device configuration information according to the values ​​of each device configuration information of the device and the corresponding weights to obtain a device resource score, determine the federated learning task code according to the mapping relationship between the stored device resource score and the machine learning algorithm, and use container technology to encapsulate the federated learning task code to obtain a mirror container.

2. The edge-oriented federated learning deployment system architecture according to claim 1, characterized in that, The module interface layer includes a RESTful-style device information upload API interface and an image container and file download API interface.

3. The edge-oriented federated learning deployment system architecture according to claim 2, characterized in that, The global center layer includes a task configuration module and a model optimization module; The task configuration module is used to configure the federated learning task code according to the edge device information transmitted by the edge terminal layer, and use container technology to encapsulate the federated learning task code to obtain a mirror container; The model optimization module is used to optimize the model parameters of the model trained on each edge device in the edge terminal layer, and to aggregate and update the model parameters.

4. The edge-oriented federated learning deployment system architecture according to claim 3, wherein The global center layer also includes an algorithm library and an image container library; The algorithm library is used to provide the federated learning task code for the task configuration module; The image container library is used to store the image container encapsulated by the task configuration module.

5. The edge federated learning deployment system architecture according to claim 2, wherein The edge terminal layer includes a Linux system building module, an image container acquisition module and a federated learning task running module; The Linux system building module is used to transplant the Bash command processor to the edge device through the terminal emulator, and pull the Linux image through the Bash command processor to install the Linux system on the edge device; The mirror container acquisition module is used to obtain and run a mirror container from the global central layer through the mirror container and the file download API interface in the root environment of the Linux system to build a running environment; The federated learning task running module is used to obtain the shell script of the federated learning task from the global central layer through the mirror container and the file download API interface in the root environment of the Linux system, and run the shell script in the running environment.

6. The edge-oriented federated learning deployment system architecture according to claim 4, wherein The container technology is Docker container technology, and the mirror container is a Docker mirror container.

7. The edge federated learning deployment system architecture according to claim 6, characterized in that, The task configuration module includes a configuration unit, a code download unit, a compilation unit, and an image generation unit; The configuration unit is used to obtain edge device information through the device information upload API interface, and determine the configuration information of the federated learning task according to the edge device information, where the configuration information includes a code download path, a code compilation instruction, and a Dockerfile; The code download unit is used to download the federated learning task code from the algorithm library according to the code download path in the Docker container; The compilation unit is used to execute the code compilation instruction to compile the federated learning task code and generate a compilation result file; The image generation unit is used to generate a Docker mirror container according to the Dockerfile and the compilation result file.

8. A method for edge federated learning deployment, characterized in that The edge-oriented federated learning deployment method is applied to the central server of the edge federated learning system. The edge federated learning system is deployed with the edge-oriented federated learning deployment system architecture as described in claim 1. The edge-oriented federated learning deployment method includes the following steps: Obtain the edge device information of the edge device through the module interface layer; the edge device information includes various device configuration information of the device; Configure the federated learning task code according to the edge device information; where, when it is determined according to the edge device information that the edge device is a resource-constrained device, select the federated learning task code corresponding to the first type of algorithm; when it is determined according to the edge device information that the edge device is not a resource-constrained device, select the federated learning task code corresponding to the second type of algorithm; the computing power requirement of the second type of algorithm is higher than that of the first type of algorithm, and the first type of algorithm is an algorithm under a lightweight machine learning framework; Wherein, the global central layer includes a task configuration module, and the task configuration module is used to perform weighted calculation on multiple device configuration information according to the values of the device configuration information of the device and the corresponding weights to obtain a device resource score, and determine the federated learning task code according to the stored mapping relationship between the device resource score and the machine learning algorithm; Encapsulate the federated learning task code using container technology to obtain a mirror container, and store the mirror container in the mirror container library; In response to an operation instruction, send the mirror container to the edge device through the module interface layer.

9. An edge-oriented federated learning deployment device, characterized in that, Includes: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, at least one of the processors implements the edge-oriented federated learning deployment method as claimed in claim 8.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that, The program executable by the processor is used to implement the edge-oriented federated learning deployment method as claimed in claim 8 when executed by the processor.

Images