AI model training method and system

Abstract

The invention provides an AI model training method and system, which are applied to a central server and a plurality of edge devices in communication connection with the central server, and the method comprises the following steps: S1, carrying out model compression and optimization on an original AI model to obtain a to-be-trained AI model; s2, carrying out containerization deployment on a to-be-trained AI model; s3, performing distributed training on the AI model to be trained by utilizing each edge device to obtain a local model parameter obtained by training each edge device; s4, aggregating the local model parameters by using the central server to obtain global model parameters after global updating; and S5, repeatedly executing the steps S3-S4 until a preset model convergence condition is met, so as to obtain a trained target AI model. According to the AI model training method and device, the model structure and the training algorithm are optimized, and AI model training is efficiently carried out on the resource-limited edge device with low power consumption.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an AI model training method and system. Background Art

[0002] With the rapid development of the Internet of Things (IoT) and mobile Internet, more and more smart devices are widely used in daily life and industrial production. The large amount of data generated by these devices needs to be analyzed and processed in real time to provide more intelligent services. Traditional centralized AI model training methods usually rely on powerful cloud computing resources, but this will lead to high network latency and bandwidth consumption, especially when processing tasks with high real-time requirements.

[0003] In recent years, edge computing has gradually attracted attention as an emerging computing paradigm. It transfers computing tasks from the central cloud to edge devices closer to the data source, which can significantly reduce the time and cost of data transmission and improve the response speed and reliability of the system. At present, there are some AI model training solutions based on edge computing, but most of them still have the following problems: 1) High resource consumption: Traditional model training methods require a lot of computing resources and memory, which is difficult to implement on edge devices with limited resources; 2) High model complexity: Although complex models have high accuracy, they are inefficient in training and reasoning on edge devices and are not suitable for real-time application scenarios; 3) High communication overhead: Frequent data transmission will increase the consumption of network bandwidth and affect the overall performance of the system.

[0004] In summary, existing edge computing platforms still face some challenges in supporting AI model training and are unable to achieve efficient and low-power model training on resource-constrained mobile devices. Summary of the invention

[0005] To this end, the present invention provides an AI model training method and system, aiming to solve the technical problem in the prior art that efficient and low-power model training cannot be achieved on resource-constrained mobile devices.

[0006] To achieve the above objectives, the present invention adopts the following technical solutions: According to a first aspect of the present invention, the present invention provides an AI model training method, which is applied to a central server and a plurality of edge devices in communication with the central server, and the method comprises: Step S1: compress and optimize the original AI model to obtain the AI ​​model to be trained; Step S2: containerize and deploy the AI ​​model to be trained so that the AI ​​model to be trained can run seamlessly on each edge device; Step S3: using each of the edge devices to perform distributed training on the AI ​​model to be trained, obtaining local model parameters obtained by training of each of the edge devices, and aggregating the multiple local model parameters to the central server; Step S4: using the central server to aggregate the local model parameters to obtain globally updated global model parameters; Step S5: Repeat the above steps S3~S4 until the preset model convergence condition is reached, that is, the trained target AI model is obtained.

[0007] Furthermore, the containerized deployment of the AI ​​model to be trained so that the AI ​​model to be trained can run seamlessly on each of the edge devices includes: Encapsulating the AI ​​model to be trained and the training environment of the AI ​​model to be trained into a lightweight Android container; Container technology is used to achieve rapid startup and resource isolation of the Android container.

[0008] Furthermore, before using each of the edge devices to perform distributed training on the AI ​​model to be trained, the method further includes: Initializing model parameters using the central server and sending the initial model parameters to all edge devices participating in the training; Each of the edge devices receives initial model parameters from the central server.

[0009] Furthermore, the distributed training of the AI ​​model to be trained by each edge device is performed to obtain local model parameters obtained by training of each edge device, and aggregating the multiple local model parameters to the central server includes: Using the local data set to perform local training on the AI ​​model to be trained, to obtain updated local model parameters; Each of the edge devices uploads the corresponding local model parameters to the central server; The central server receives the local model parameters uploaded by each of the edge devices.

[0010] Furthermore, the locally training the AI ​​model to be trained using the local data set includes: Use a gradient descent algorithm to perform local training on the AI ​​model to be trained; The local data set and the current model parameters are used to perform forward propagation and backward propagation to calculate new local model parameters.

[0011] Furthermore, the aggregating the local model parameters by the central server to obtain globally updated global model parameters includes: The central server uses an aggregation algorithm to merge all local model parameters into new global model parameters, which are expressed as follows: W_new=sum(n_i*W_i') / sum(n_i) Among them, W_new is the new global model parameter; n_i is the data volume of the i-th edge device; W_i' is the local model parameter of the i-th edge device.

[0012] Furthermore, after obtaining the globally updated global model parameters, the method further includes: The central server broadcasts the global model parameters to all edge devices participating in the training.

[0013] Furthermore, the method further comprises: Applying the global model parameters to the AI ​​model to be trained, and evaluating the model performance parameters of the AI ​​model to be trained using the central server and / or some edge devices; the model performance parameters include at least one of a loss function and an accuracy rate; Determine whether the AI ​​model to be trained meets the preset model convergence conditions based on the model performance parameters.

[0014] According to a second aspect of the present invention, the present invention provides an AI model training system, the system comprising a central server and a plurality of edge devices communicatively connected to the central server; The central server and the edge device are used to execute the AI ​​model training method as described in any one of the first aspects of the present invention.

[0015] The present invention adopts the above technical solution and has at least the following beneficial effects: Through the present invention, a method for implementing lightweight AI model training based on an edge Android container platform is proposed, wherein the AI ​​model to be trained is obtained by compressing and optimizing the original AI model; the AI ​​model to be trained is containerized and deployed so that the AI ​​model to be trained can run seamlessly on each edge device; the AI ​​model to be trained is distributedly trained by each edge device to obtain local model parameters obtained by training each edge device, and multiple local model parameters are aggregated to the central server; the local model parameters are aggregated by the central server to obtain global model parameters after global update; the distributed training and aggregation processing are repeated until the global model parameters reach the preset model convergence conditions to obtain the target AI model that has been trained. Thus, by optimizing the model structure and training algorithm, efficient and low-power AI model training can be achieved on resource-constrained edge devices.

[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of a process flow of an AI model training method provided by an embodiment of the present invention is shown; Figure 2 A schematic diagram showing a flow chart of an AI model training method provided by another embodiment of the present invention is shown; Figure 3 A structural diagram of an AI model training system provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0019] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0020] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0021] The embodiment of the present invention provides an AI model training method, such as Figure 1 As shown, the application to the central server and multiple edge devices connected to the central server for communication may at least include the following steps S101-S105: Step S101, compressing and optimizing the original AI model to obtain the AI ​​model to be trained.

[0022] For the original AI model to be trained, model pruning, quantization and other technologies can be used to compress the model to reduce the number of parameters and computational complexity of the original AI model. After model compression, the knowledge of the large model can be transferred to the small model through knowledge distillation technology to improve the performance of the small model, thereby obtaining a compressed and optimized AI model to be trained.

[0023] Step S102: Containerize and deploy the AI ​​model to be trained so that the AI ​​model to be trained can run seamlessly on each edge device.

[0024] In an embodiment of the present invention, the AI ​​model to be trained and its training environment are packaged together into a lightweight Android container to ensure that the AI ​​model to be trained can run seamlessly on different edge devices. During operation, container technology can be used to achieve rapid startup and resource isolation of the Android container. Preferably, the container technology can be Docker technology.

[0025] Step S103: Use each edge device to perform distributed training on the AI ​​model to be trained, obtain local model parameters obtained by training each edge device, and summarize multiple local model parameters to the central server.

[0026] Specifically, the embodiment of the present invention uses federated learning technology to combine the data and computing resources on multiple edge devices to jointly participate in model training. For the AI ​​model to be trained, local training is performed on each edge device, and then the model parameters are aggregated to the central server for global update.

[0027] like Figure 2 As shown, before using each edge device to perform distributed training on the AI ​​model to be trained, the embodiment of the present invention also needs to use the central server to initialize the model parameters and send the initial model parameters W_0 to all edge devices participating in the training. Then, each edge device receives the initial model parameters W_0 from the central server, uses the local data set to perform local training on the AI ​​model to be trained, and obtains the updated local model parameters.

[0028] Specifically, the Mini-batch Gradient Descent algorithm or the Stochastic Gradient Descent algorithm can be used to perform local training on the AI ​​model to be trained, and the local data set and the current model parameters Wi are used for forward propagation and back propagation to calculate the new local model parameters Wi'.

[0029] Furthermore, each edge device may upload the corresponding local model parameter W_i' to the central server, and the central server receives the local model parameter W_i' uploaded from each edge device and performs a global update operation.

[0030] Step S104, using the central server to aggregate the local model parameters to obtain globally updated global model parameters.

[0031] Specifically, the central server can use an aggregation algorithm to merge local model parameters into new global model parameters, which can be expressed as follows: W_new=sum(n_i*W_i') / sum(n_i) Among them, W_new is the new global model parameter; n_i is the data volume of the i-th edge device; W_i' is the local model parameter of the i-th edge device.

[0032] Preferably, the aggregation algorithm can use a weighted average algorithm, that is, weighting according to the amount of data of each edge device; other aggregation methods suitable for different scenarios can also be used, such as median aggregation, trimmed mean, etc.

[0033] In the embodiment of the present invention, after obtaining the global model parameters, the central server broadcasts the global model parameters W_new to all edge devices participating in the training.

[0034] Step S105, repeat the above steps S103~S104 until the preset model convergence condition is reached, that is, the trained target AI model is obtained.

[0035] That is to say, the embodiment of the present invention utilizes multiple edge devices to iteratively train the AI ​​model to be trained based on a distributed training method until a preset number of iterations or model convergence conditions are reached.

[0036] Specifically, after each training round, the global model parameters can be applied to the AI ​​model to be trained, and the central server can be used to centrally evaluate the model performance, or some edge devices can be selected to perform distributed evaluation of the model performance, and the model performance parameters can be evaluated based on the validation set, including loss function, accuracy, etc., to determine whether the model convergence conditions are met. When the training is completed, the central server saves the final global model parameters W_final, and the target AI model is obtained.

[0037] Furthermore, in the distributed training process, the embodiment of the present invention can obtain the real-time resource status of each edge device in real time; according to the real-time resource status, the batch size and learning rate of the distributed training can be dynamically adjusted to ensure the stability and efficiency of the training process. In addition, the embodiment of the present invention can also use a resource scheduling algorithm to optimize the resource allocation of the Android container to improve the overall performance of the training method.

[0038] Furthermore, in the data transmission process between the central server and each edge device, the embodiment of the present invention can also adopt a variety of secure transmission technologies to ensure the security of data transmission. Among them, the secure transmission technology can include implementing fine-grained data access control technology to ensure the security of data during transmission and storage; differential privacy technology to protect the privacy of user data and prevent sensitive information leakage; secure transmission protocols, such as HTTPS to ensure the security of parameter transmission and incremental transmission technology, that is, only uploading the difference part of the parameter to reduce the transmission volume; data encryption mechanism, etc.

[0039] The embodiment of the present invention provides an AI model training method, which realizes distributed training based on a central server and multiple edge devices, combining the advantages of centralized training (transmitting data to the central cloud for model training, and then sending the trained model to the edge device, which is suitable for scenarios with small data volume and good network conditions, but will increase network bandwidth consumption and delay), local training (model training on a single edge device, independent of other devices or the central cloud, suitable for edge devices with relatively rich resources, but the training efficiency and model performance may be low), and hybrid training (combining the advantages of edge computing and cloud computing, offloading some computing tasks to the central cloud, and executing some tasks on the edge device, which can balance resource consumption and training efficiency to a certain extent, but the implementation complexity is relatively high), and overcomes the defects of each training method, and realizes efficient and low-power AI model training on resource-constrained edge devices. In addition, the present invention has at least the following advantages: 1) Efficiency: Through model compression and optimization, the resource consumption of the model is significantly reduced and the training efficiency is improved; 2) Low power consumption: Lightweight models and containerized deployment ensure cross-device compatibility and portability, enabling edge devices to perform model training in a low-power state.

[0040] 3) Real-time: Through the coordinated distributed training of multiple edge devices and the adaptive resource management that dynamically adjusts the training parameters according to the real-time resource status of the edge devices, the real-time response capability of the system is ensured, which is suitable for various real-time application scenarios; 4) Security: Through real-time fine-grained data access control and differential privacy technology, the security and privacy of user data are effectively protected.

[0041] 5) Scalability: Supports multiple types of edge devices and can be easily expanded to more application scenarios.

[0042] Further, as Figure 1 The specific implementation of the present invention provides an AI model training system, such as Figure 3As shown, the system may include: a central server 310 and a plurality of edge devices 320 communicatively connected to the central server 310 .

[0043] The central server 310 can be used to compress and optimize the original AI model to obtain the AI ​​model to be trained; containerize and deploy the AI ​​model to be trained so that the AI ​​model to be trained can run seamlessly on each edge device 320; initialize model parameters and send the initial model parameters to all edge devices 320 participating in the training; The edge device 320 can be used to receive the initial model parameters from the central server 310, and repeatedly perform the following steps until the global model parameters reach the preset model convergence condition, and obtain the trained target AI model: perform distributed training on the AI ​​model to be trained, obtain the local model parameters obtained by the training of each edge device 320, and aggregate the multiple local model parameters to the central server 310; The central server 310 can also be used to repeatedly perform the following steps until the global model parameters reach the preset model convergence conditions to obtain a trained target AI model: aggregate the local model parameters to obtain the globally updated global model parameters; broadcast the global model parameters to all edge devices 320 participating in the training.

[0044] Optionally, the central server 310 may also be used to encapsulate the AI ​​model to be trained and the training environment of the AI ​​model to be trained into a lightweight Android container; Container technology is used to achieve fast startup and resource isolation of Android containers.

[0045] Optionally, the edge device 320 may also be used to perform local training on the AI ​​model to be trained using a local data set to obtain updated local model parameters; The corresponding local model parameters are uploaded to the central server 310 .

[0046] Optionally, the edge device 320 may also be used to perform local training on the AI ​​model to be trained using a gradient descent algorithm; Use the local dataset and current model parameters to perform forward propagation and back propagation to calculate new local model parameters.

[0047] Optionally, the central server 310 may also be used to receive local model parameters uploaded by each edge device 320, and use an aggregation algorithm to merge all local model parameters into new global model parameters, which are expressed as follows: W_new=sum(n_i*W_i') / sum(n_i) Wherein, W_new is a new global model parameter; n_i is the data volume of the i-th edge device 320; W_i' is a local model parameter of the i-th edge device 320; The global model parameters are broadcast to all edge devices 320 participating in the training.

[0048] Optionally, the central server 310 and / or the edge device 320 may also be used to apply the global model parameters to the AI ​​model to be trained, and evaluate the model performance parameters of the AI ​​model to be trained; the model performance parameters include at least one of the loss function and the accuracy rate; Determine whether the AI ​​model to be trained meets the preset model convergence conditions based on the model performance parameters.

[0049] Optionally, the central server 310 may also be used to obtain the real-time resource status of each edge device 320 in real time during the distributed training process; Dynamically adjust the batch size and / or learning rate of distributed training based on real-time resource conditions; and / or, Optimize resource allocation of Android containers using resource scheduling algorithms.

[0050] Optionally, during the data transmission process between the central server 310 and each edge device 320, a secure transmission technology is used to ensure the security of data transmission; Among them, the secure transmission technology includes implementing at least one of fine-grained data access control technology, data encryption mechanism, differential privacy technology, secure transmission protocol and incremental transmission technology.

[0051] It should be noted that for other corresponding descriptions of the functional modules involved in the AI ​​model training system provided in the embodiment of the present invention, please refer to Figure 1 The corresponding description of the method shown will not be repeated here.

[0052] Those skilled in the art can clearly understand that the specific working processes of the systems, devices, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and for the sake of brevity, they are not further described here.

[0053] In addition, the functional units in various embodiments of the present invention may be physically independent of each other, or two or more functional units may be integrated together, or all functional units may be integrated into one processing unit. The above integrated functional units may be implemented in the form of hardware, or in the form of software or firmware.

[0054] A person skilled in the art can understand that if the integrated functional unit is implemented in the form of software and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, which includes several instructions to enable a computing device (such as a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention when running the instructions. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0055] Alternatively, all or part of the steps of implementing the aforementioned method embodiments may be accomplished by hardware associated with program instructions (such as a computing device such as a personal computer, a server, or a network device), and the program instructions may be stored in a computer-readable storage medium. When the program instructions are executed by a processor of the computing device, the computing device executes all or part of the steps of the method described in the various embodiments of the present invention.

[0056] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that within the spirit and principles of the present invention, the technical solutions described in the aforementioned embodiments can still be modified, or some or all of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate from the protection scope of the present invention.

Claims

1. An AI model training method, characterized in that: Applied to a central server and a plurality of edge devices in communication with the central server, the method comprises: Step S1: compress and optimize the original AI model to obtain the AI ​​model to be trained; Step S2: containerize and deploy the AI ​​model to be trained so that the AI ​​model to be trained can run seamlessly on each edge device; Step S3: using each of the edge devices to perform distributed training on the AI ​​model to be trained, obtaining local model parameters obtained by training of each of the edge devices, and aggregating the multiple local model parameters to the central server; Step S4: using the central server to aggregate the local model parameters to obtain globally updated global model parameters; Step S5: Repeat the above steps S3~S4 until the preset model convergence condition is reached, that is, the trained target AI model is obtained.

2. The method according to claim 1, characterized in that The containerized deployment of the AI ​​model to be trained so that the AI ​​model to be trained can run seamlessly on each edge device includes: Encapsulating the AI ​​model to be trained and the training environment of the AI ​​model to be trained into a lightweight Android container; Container technology is used to achieve rapid startup and resource isolation of the Android container.

3. The method according to claim 1, characterized in that Before using each of the edge devices to perform distributed training on the AI ​​model to be trained, the method further includes: Initializing model parameters using the central server and sending the initial model parameters to all edge devices participating in the training; Each of the edge devices receives initial model parameters from the central server.

4. The method according to claim 3, characterized in that The method of using each of the edge devices to perform distributed training on the AI ​​model to be trained, obtaining local model parameters obtained by training each of the edge devices, and aggregating the multiple local model parameters to the central server includes: Using the local data set to perform local training on the AI ​​model to be trained, to obtain updated local model parameters; Each of the edge devices uploads the corresponding local model parameters to the central server; The central server receives the local model parameters uploaded by each of the edge devices.

5. The method according to claim 4, characterized in that The locally training the AI ​​model to be trained using the local data set includes: Use a gradient descent algorithm to perform local training on the AI ​​model to be trained; The local data set and the current model parameters are used to perform forward propagation and backward propagation to calculate new local model parameters.

6. The method according to claim 3, characterized in that The method of aggregating the local model parameters by the central server to obtain globally updated global model parameters includes: The central server uses an aggregation algorithm to merge all local model parameters into new global model parameters, which are expressed as follows: W_new=sum(n_i*W_i') / sum(n_i) Among them, W_new is the new global model parameter; n_i is the data volume of the i-th edge device; W_i' is the local model parameter of the i-th edge device.

7. The method according to claim 1, characterized in that After obtaining the globally updated global model parameters, the method further includes: The central server broadcasts the global model parameters to all edge devices participating in the training.

8. The method according to claim 7, characterized in that The method further comprises: Applying the global model parameters to the AI ​​model to be trained, and evaluating the model performance parameters of the AI ​​model to be trained using the central server and / or some edge devices; the model performance parameters include at least one of a loss function and an accuracy rate; Determine whether the AI ​​model to be trained meets the preset model convergence conditions based on the model performance parameters.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: During the distributed training process, real-time resource status of each edge device is obtained in real time; Dynamically adjust the batch size and / or learning rate of distributed training according to the real-time resource status; and / or, Optimize resource allocation of Android containers using resource scheduling algorithms; and / or, During the data transmission process between the central server and each of the edge devices, secure transmission technology is used to ensure the security of data transmission; Among them, the secure transmission technology includes implementing at least one of fine-grained data access control technology, data encryption mechanism, differential privacy technology, secure transmission protocol and incremental transmission technology.

10. An AI model training system, characterized in that: The system includes a central server and a plurality of edge devices communicatively connected to the central server; The central server and the edge device are used to execute the AI ​​model training method as described in any one of claims 1 to 9.

Images