Distributed training optimization method and main operation device

By using parameter training of local models and whole-domain models in distributed training, deviating parameters are identified and updated, the model misjudgment problem caused by data pollution is solved, and the accuracy of the artificial intelligence model and the stability of the production line are improved.

CN119026656APending Publication Date: 2024-11-26WISTRON CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310696381.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-23
Filing Date
2023-06-13
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

In artificial intelligence model training, data pollution leads to misjudgment of models, and it is difficult for existing technology to effectively identify and deal with data pollution problems, affecting production efficiency and product quality.

Method used

By using parameter training of local models and global models of multiple sample sets in distributed training, the deviation parameters are judged and updated, and contaminated data are identified and excluded, thereby improving the accuracy of the model.

Benefits of technology

Effectively identifying and handling data pollution improves the accuracy of artificial intelligence models, reduces the rate of misjudgment, and improves the stability of the production line and product quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119026656A_ABST
    Figure CN119026656A_ABST
Patent Text Reader

Abstract

The invention provides a distributed training optimization method and a main operation device. In the method, a local model is trained using one of the plurality of sample sets and global parameters of the global model to generate local parameters of the local model. One or more deviation parameters of local parameters of a plurality of local models trained by a plurality of sample sets are determined. The distribution of deviation parameters is remote from the distribution of other local parameters, and the local parameters of these local models are used to update the global parameters of the global model. Therefore, the prediction accuracy of the global model can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a distributed training technology, and in particular to an optimization method and a main computing device for distributed training. Background Art

[0002] In order to improve production efficiency and product quality, companies are gradually introducing artificial intelligence (AI) technology into the factory. For example, through AI defect detection. Before the AI ​​model can be tested, it must first collect factory image data to train the model, and then deploy the model on the production line after training. A product may be produced by multiple production lines. If the production line of this product is to introduce AI technology, the training data of the AI ​​model will be collected from these production lines. Each production line may be located in a different factory location, and the factories may be all over the world. After collecting the data of these production lines, the model training is performed separately for the data of each production line.

[0003] However, if the production line data is incorrect, it will affect the trained AI model and cause production problems. Therefore, the accuracy of the AI ​​model is closely related to the collected data. If the data provided by the production line or factory is contaminated, the trained AI model will not be able to make accurate judgments and make misjudgments. Data pollution caused by incorrect data may be that the data is incorrectly labeled or injected with data that is not for the product. For example, a picture of solder inspection of surface mount technique (SMT) is actually marked as empty soldering when there is no empty soldering. If such a marked picture is used to train the AI ​​model, the trained AI model will make misjudgments, which will cause problems on the production line.

[0004] Of course, in addition to AI applications in production lines, other application scenarios may also have data pollution problems. Therefore, how to eliminate the problem of data pollution is also a key technology for the relevant industries to introduce AI. Summary of the invention

[0005] The embodiment of the present invention provides a distributed training optimization method and a main computing device, which can identify contaminated data.

[0006] The optimization method of distributed training of the embodiment of the present invention includes (but is not limited to) training a local model by using one of a plurality of sample sets and the global parameters of the global model to generate local parameters of the local model. Determining one or more deviation parameters in the local parameters of the plurality of local models trained by the plurality of sample sets. The distribution of the deviation parameters is far from the distribution of other local parameters, and the local parameters of these local models are used to update the global parameters of the global model.

[0007] The main computing device of the embodiment of the present invention includes (but is not limited to) a memory and a processor. The memory stores program code. The processor is coupled to the memory. The processor loads the program code to execute: obtaining local parameters corresponding to multiple sample sets, and determining one or more deviation parameters in the local parameters of multiple local models trained by these sample sets. The local parameters corresponding to each sample set are generated by training the local model using one of those sample sets and the global parameters of the global model. The distribution of the deviation parameters is far away from the distribution of other local parameters, and the local parameters of these local models are used to update the global parameters of the global model.

[0008] Based on the above, according to the distributed training optimization method and the main computing device of the embodiment of the present invention, it is determined whether the local parameter is a deviation parameter based on the distribution of the local parameter of the local model. Thus, it is possible to determine which data is contaminated, and thus deal with the contaminated data (i.e., deviation parameters), thereby improving the accuracy of the trained model.

[0009] In order to make the above features and advantages of the present invention more clearly understood, embodiments are given below with reference to the accompanying drawings for detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a schematic diagram of a training system according to an embodiment of the present invention;

[0011] Figure 2 is a flow chart of a distributed training optimization method according to an embodiment of the present invention;

[0012] Figure 3 is a schematic diagram illustrating data distribution according to an embodiment of the present invention;

[0013] Figure 4 is a flow chart of pollution determination according to an embodiment of the present invention;

[0014] Figure 5 FIG. 1 is a schematic diagram illustrating using an autoencoder to identify deviation parameters according to an embodiment of the present invention.

[0015] Explanation of symbols

[0016] 1: Training system

[0017] 10: Main computing device

[0018] 20-1~20-n: Secondary computing device

[0019] 30-1~30-n: Database

[0020] 11: Memory

[0021] 12: Communication transceiver

[0022] 13: Processor

[0023] S210~S220, S410~S430: Steps

[0024] Weight Parameters

[0025] w: weight value

[0026] g. Gradient Value

[0027] Reconstructing data DETAILED DESCRIPTION

[0028] Figure 1 is a schematic diagram of a training system 1 according to an embodiment of the present invention. Figure 1 The training system 1 includes (but is not limited to) a main computing device 10, secondary computing devices 20-1 to 20-n (r and n are positive integers, and r is greater than 1 but less than n) and databases 30-1 to 30-n.

[0029] The main computing device 10 and / or the secondary computing devices 20 - 1 ˜ 20 - n may be one or more desktop computers, notebook computers, smart phones, tablet computers, wearable devices, servers, smart assistants, cloud platforms or other electronic devices.

[0030] The main computing device 10 includes (but is not limited to) a memory 11, a communication transceiver 12, and a processor 13. In one embodiment, the main computing device 10 includes a plurality of servers, each of which includes a memory, a communication transceiver, and a processor.

[0031] The memory 11 may be any type of fixed or removable random access memory (RAM), read only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD) or similar element. In one embodiment, the memory 11 is used to store program code, software modules, configurations, data or files (e.g., data, model parameters, or calculation values), and will be described in detail in subsequent embodiments.

[0032] The communication transceiver 12 may be a communication transceiver circuit or a transmission interface card supporting Wi-Fi, Bluetooth, mobile communication, USB or Ethernet. In one embodiment, the communication transceiver 12 is used to transmit or receive data with an external device (e.g., the secondary computing devices 20-1 to 20-n or the databases 30-1 to 30-n).

[0033] The processor 13 is coupled to the memory 11 and the communication transceiver 12. The processor 13 may be a central processing unit (CPU), a graphics processing unit (GPU), or other programmable general-purpose or special-purpose microprocessors, digital signal processors (DSP), programmable controllers, field programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), neural network accelerators, or other similar components or combinations of the above components. In one embodiment, the processor 13 is used to execute all or part of the operations of the main computing device 10, and can load and execute various program codes, software modules, files, and data stored in the memory 11.

[0034] The secondary computing devices 20 - 1 ˜ 20 - n may include (but are not limited to) a memory 11 , a communication transceiver 12 , and a processor 13 that are the same as or compatible with the primary computing device 10 , and the implementation and function of each component will not be described in detail.

[0035] The databases 30-1 to 30-n are connected to the secondary computing devices 20-1 to 20-n. The databases 30-1 to 30-n can be a flash drive, a memory, a hard disk, or a network hard disk. In one embodiment, the databases 30-1 to 30-n are used to store sample sets for training. For example, pictures and labeling results. The labeling results can be objects, locations, and events in the pictures. Another example is sounds and labeling results. The classification results can be the source type, speech, or song corresponding to the sound. However, the content of the samples in the sample set can still be adjusted according to actual needs.

[0036] Hereinafter, the method described in the embodiment of the present invention will be described in conjunction with various devices, components and modules in the training system 1. Each process of the method can be adjusted according to the implementation situation, and is not limited thereto.

[0037] Usually, before training an AI model, data must be collected as a sample set for training. Taking the production line as an example, in order to introduce AI technology for production, the factory must collect a large amount of production line data before starting model training. Before model training, the data will be pre-processed and classified and labeled. However, model training may encounter errors. For example, incorrect labeling of data causes an untrustworthy model to be trained, making it impossible for the AI ​​model to make accurate judgments. This type of labeling error can be regarded as a form of data pollution. Therefore, two problems may be faced during training: First, it is impossible to determine whether the data is contaminated, resulting in the training of a poor quality model, and the model cannot make accurate predictions; second, it is impossible to determine at which time point and which source the data is contaminated. In order to solve such problems, data contamination judgment can be introduced into distributed training to detect contaminated data and then train a reliable model.

[0038] Figure 2 is a flow chart of an optimization method for distributed training according to an embodiment of the present invention. Figure 2 , each time the computing device 20-1~20-n trains the local model by using one of the multiple sample sets and the global parameters of the global model to generate local parameters of the local model (step S210). Specifically, in the architecture of data parallel distributed training, the trained model parameters (e.g., the global weight of the global model) are estimated and stored by the main (master) device (e.g., the main computing device 10), and the data is trained by a group of worker devices (e.g., the secondary computing devices 20-1~20-n). When a set of training data is collected, the training data may be first distributed into multiple slices (shards), and then these slices are distributed to each worker device. The worker devices are each trained based on the allocated slices / training data, and then their trained model parameters are sent to the main device. The main device is responsible for integrating these model parameters into a global parameter (e.g., the global weight), and then forming the final model.

[0039] For example, in the program code snippet (1) shown below, the worker device (i.e., worker) will first download the corresponding data slice and download the global weight parameters for training, and then transmit the results of each training to the main device (i.e., master) for parameter integration to obtain the global weight for the next training. For distributed training, taking the data applied to the factory production line as an example, it is usually impossible to determine which data slice has a problem during training. Therefore, the embodiments of the present invention provide improvements for distributed training.

[0040] Program code snippet (1):

[0041]

[0042]

[0043] In one embodiment, each computing device 20-1 to 20-n can train a local model based on a machine learning algorithm. The machine learning algorithm is, for example, YOLO, a convolutional neural network (CNN), a long short-term memory (LSTM), or a generative adversarial network (GAN), but is not limited thereto. The machine learning algorithm can analyze the relationship between the training sample and the corresponding label or real result to obtain the law, thereby predicting the unknown data through the law. Each local model is a machine learning model constructed by each computing device 20-1 to 20-n after (supervised) learning using a sample set, and is used to infer the data to be evaluated. The sample set includes data / data slices and their corresponding labeling results. For example, a picture (i.e., data) and the type of object in the picture (i.e., labeling result). For another example, a sound (i.e., data) and the person who speaks it (i.e., labeling result). Multiple sample sets can be distinguished based on source, object and / or time. For example, the first sample set is a sample set of the first production line at the first time point, and the second sample set is a sample set of the second production line at the first time point. For another example, the third sample set is a sample set of the third production line at the second time point, and the fourth sample set is a sample set of the fourth production line at the third time point.

[0044] It should be noted that this local model can be used for image recognition / classification, object detection, semantic analysis or other inferences, and the embodiments of the present invention do not limit its use. In some application scenarios, the trained local model may reach a preset accuracy standard. Local parameters are parameters in the local model. For example, gradient values / vectors, weight parameters in a neural network, bias parameters, or parameters used in other machine learning algorithms. In other words, each local model is a machine learning model formed based on local parameters.

[0045] The global model is a model derived by the main computing device 10 based on the local parameters of multiple local models. The global parameters are the parameters in the global model. For example, gradient values / vectors, weight parameters in neural networks, bias parameters or parameters used in other machine learning algorithms. In other words, the global model is a machine learning model formed by global parameters derived from local parameters of multiple local models. This global model can be used for image recognition / classification, object detection, semantic analysis or other inferences, and the embodiments of the present invention do not limit its use. As shown in the program code snippet (1), each computing device 20-1~20-n will load the global parameters used for the current training as a pre-trained model. Each computing device 20-1~20-n can use a sample set to train the pre-trained model. The local parameters obtained from the training can be used to correct / update the global parameters used for the current training.

[0046] Next, the processor 13 of the main computing device 10 can obtain the local parameters corresponding to the multiple sample sets through the communication transceiver 12. As described above, the local parameters corresponding to each sample set are generated each time the computing devices 20-1 to 20-n train the local model based on the machine learning algorithm using one of the sample sets and the global parameters of the global model. Alternatively, the processor 13 loads the multiple local parameters corresponding to the multiple sample sets from the memory 11.

[0047] Please refer to Figure 2 , the processor 13 of the main computing device 10 can determine one or more deviation parameters in the local parameters of the multiple local models trained by the multiple sample sets (step S220). Specifically, as described above, the local parameters obtained in each training can be used to correct / update the global parameters used in the current training. That is, the local parameters of these local models are used to update the global parameters of the global model.

[0048] On the other hand, when the labeling is wrong (i.e., the labeling result is different from the actual situation of the data), the sample set will be contaminated, and the local parameters or even the global parameters derived based on this set will also be contaminated. In the embodiment of the present invention, these contaminated local parameters are called deviation parameters. It has been experimentally proved that the distribution of deviation parameters is far away from the distribution of other local parameters. Distribution refers to the statistics of multiple parameters at multiple values.

[0049] For example, Figure 3 is a schematic diagram illustrating data distribution according to an embodiment of the present invention. Figure 3 , the global parameter is weighted (For example, the weights used by neurons in a neural network to perform weighted operations on feature values; w is the weight value, i t For example, the weight parameter of the global model trained by the contaminated sample set at the tth time point in the i-th training is The distribution of (upper left) will be different from the weight parameter obtained for the uncontaminated sample set The distribution of the upper right figure will be different. The distribution shown in the upper right figure can distinguish two groups, and the group occupying a smaller portion is far away from the group occupying a larger portion. In the embodiment of the present invention, the parameter of the group occupying a smaller portion is called a deviation parameter.

[0050] It is worth noting that the weight parameter The gradient value related to the tth time point in the i-th training And w (t) is a fixed weight value. In one embodiment, during machine learning, the smaller the value of the loss function (or cost function) and / or the objective function, the better. The smaller the value of the loss function, the higher the accuracy of the model. The gradient descent method is an algorithm (there may also be exponentially weighted average method, momentum gradient descent method or other optimization algorithms) to correct the local parameters of the local model (that is, update the loaded global parameters) based on the result of the loss function, so as to find the local parameters with smaller or minimum values ​​of the loss function. The gradient value is the difference between the local / global parameters obtained in this training and the local parameters with smaller or minimum values ​​of the loss function. The gradient vector corresponds to a set of differences between multiple local / global parameters.

[0051] Because of w (t) is a fixed weight value, so the gradient value of the global model trained by the contaminated sample set is The distribution of (lower left) will be different from the weight parameter obtained for the uncontaminated sample set The distribution of (lower right figure) will also be different. Based on the characteristics of such distribution deviation / distant, it is possible to identify whether the sample set is contaminated or whether the local parameter is a deviation parameter.

[0052] In one embodiment, the local parameters of each local model include local correction parameters, the global parameters include global correction parameters, and the global correction parameters are obtained based on the local correction parameters through weighted operations. For example, the local correction parameters are gradient values ​​for the local model. That is, the local correction parameters are gradient values ​​determined based on the gradient descent method. In one embodiment, the local correction parameters can be distinguished based on the time point of acquisition / generation / or transmission. The global correction parameters are gradient values ​​for the global model. The sum of the products of the local correction parameters of multiple local models and the corresponding operation weights (i.e., weighted operations; for example, α r is the calculation weight of the local correction parameter of the r-th calculation device 20-r, is the gradient value (i.e., local correction parameter) of the r-th computing device 20-r in the i-th training, which can be used to determine the global correction parameter

[0053] The processor 13 of the main computing device 10 can determine the deviation parameter according to the operation weight used in the weighted operation of the local correction parameter of each local model. r The weight parameter The data distribution of is considered as Gaussian distribution. Relative to the gradient value And w (t) is a fixed model parameter, so when the weight parameter When the data distribution is considered as Gaussian distribution, the gradient value The data distribution can also be considered as Gaussian distribution.

[0054] In one embodiment, the computation weight α r The mathematical function f() is:

[0055]

[0056] , where n The average n The number of variance T refers to the inverse sign of the matrix in linear algebra, and is the exponential part of the Gaussian distribution. μ c The closer it is, the greater its α r The larger the value, the greater the influence of the corresponding data on the final weight; on the contrary, μ c The farther away, the r The value will be smaller.

[0057] It is worth noting that the local correction parameters (e.g., the gradient value of the local model) obtained by training with the contaminated sample set will be ) into equation (1) will be smaller than the weight corresponding to the local correction parameter obtained by training with the uncontaminated sample set. In other words, the main computing device 10 calculates the global parameters (e.g., weight parameter w (it) ), the influence of the local correction parameters trained using the contaminated sample set will be relatively small. Therefore, the influence of the deviated parameters or the contaminated sample set on the global parameters can be reduced by a smaller calculation weight, thereby obtaining a reliable global model.

[0058] How to know that the local correction parameter has a smaller computational weight? Figure 4This is a flow chart of pollution determination according to an embodiment of the present invention. Figure 4 , the processor 13 may compare the operation weight used in the weighted operation of the local correction parameter of each local model with the deviation threshold value (step S410). That is, the deviation threshold value is used as a threshold value to determine whether the operation weight is small, and is used to determine whether the data is contaminated. In response to the operation weight being less than the deviation threshold value, the processor 13 may determine that the local correction parameter has a smaller operation weight, and use the local correction parameter of the local model as a deviation parameter (step S420). In response to the operation weight not being less than the deviation threshold value, the processor 13 may determine that the local correction parameter has a larger operation weight than the deviation parameter, and prohibit / not use the local correction parameter of the local model as a deviation parameter (step S430). In one embodiment, in response to the operation weight not being less than the deviation threshold value, the processor 13 may issue an alarm. For example, the responsible person is notified of an alarm in a visual or auditory form.

[0059] In one embodiment, the processor 13 may determine the probability distribution of the operation weights used in the weighted operation of the local correction parameters of the multiple local models. Since these operation weights are unknown parameters, in one embodiment, the probability distribution is a t distribution (or Student's t-distribution). In probability theory and statistics, the t distribution is used to estimate the expected value of a normal distribution with an unknown standard deviation based on a small sample.

[0060] The processor 130 may use the value corresponding to the deviation quantile in the probability distribution as the deviation

[0061] Distribution function (Cumulative Distribution Function, CDF), then P(α r <b)=P(s<

[0062]

[0063] The program code snippet (2) is an example to illustrate the aforementioned technical solution based on calculation weights:

[0064]

[0065]

[0066] In one embodiment, for the corresponding treatment of the deviation parameter, the processor 13 may warn the deviation parameter. For example, the communication transceiver 12 notifies the secondary computing device that outputs the deviation parameter about the problem of marking error. For another example, a display (not shown) presents a visual warning.

[0067] In another embodiment, the local parameters of each local model include local correction parameters, and the global parameters include global correction parameters. For example, the local correction parameters are gradient values ​​for the local model. That is, the local correction parameters are gradient values ​​determined based on the gradient descent method. And the global correction parameters are gradient values ​​for the global model. Figure 3 It can be understood that the weight parameters / gradient values ​​trained by the contaminated sample set have different numerical distributions from the uncontaminated weight parameters / gradient values. In other words, the characteristics of the model parameters generated by the contaminated and uncontaminated sample sets are different and can be classified into two groups. Therefore, if you want to distinguish whether the data is contaminated, you can regard this problem as a binary classification problem.

[0068] The processor 13 of the main computing device 10 can determine whether the local correction parameter of each local model is a deviation parameter through a detection model. The detection model is a model trained by a machine learning algorithm using known local correction parameters and known labeling results (for example, deviation parameters or non-deviation parameters). The machine learning algorithm can be an autoencoder, a decision tree, KNN (K Nearest Neighbor), a support vector machine (Support Vector Machine, SVM), a random forest, or a neural network (NN), but is not limited thereto. The detection model is used to identify whether the local correction parameter is a deviation parameter. The deviation parameter is, for example, a local gradient value / vector that is not suitable for calculating the weight parameter.

[0069] For example, Figure 5 is a schematic diagram illustrating the use of an autoencoder to identify deviation parameters according to an embodiment of the present invention. Figure 5 The error detection model based on the auto encoder includes an encoder 501 and a decoder 502. The error detection model is a model trained using the gradient values ​​of all non-deviation parameters. Assume that the input data is the gradient value obtained by the r-th computing device 20-r at the t-th time point in the i-th training. Compressing data is the gradient value The encoder 501 of the error detection model is compressed into data with a relatively low dimension. Then, the decoder 501 can convert the compressed data into Decompress to reconstruct data

[0070] React to | The gradient value Can be used to calculate weight parameters. The gradient value It can be considered as a deviation parameter, and even this gradient value can be excluded from the local parameters used to calculate the global parameters To avoid the contaminated deviation parameters from affecting the global parameters. The value of a can be determined by all It is found at a specific quantile (for example, 98, 97.5, or 95) of the sample distribution of .

[0071] It should be noted that in other embodiments, the local parameters or local modified parameters marked as deviation parameters and the local modified parameters marked as non-deviation parameters may also be used simultaneously to train the error detection model.

[0072] In one embodiment, the processor 13 of the main computing device 10 may delete the deviation parameters from the local parameters of the multiple local models trained by the multiple sample sets, and only use other local parameters that are not deviation parameters to update the global parameters of the global model. For example, the local parameters from the r-th computing device 20-r are deviation parameters. The main computing device 10 may ignore the local parameters of the r-th computing device 20-r, and determine the global parameters of the global model based only on the local parameters of the 1st to r-1th computing devices and the r+1th to nth computing devices.

[0073] The program code snippet (3) is an example to illustrate the above technical solution based on the error detection model:

[0074]

[0075]

[0076]

[0077] In summary, the optimization method and main computing device of the distributed training of the embodiment of the present invention can identify the contaminated deviation parameters through the operation weight or detection model based on the characteristics of the numerical distribution deviation, and reduce the proportion of the deviation parameters in the calculation of the global parameters or directly exclude the deviation parameters for calculating the global parameters. In this way, data contamination can be reduced or avoided, thereby increasing the prediction accuracy of the global model.

[0078] In some application scenarios, taking factories as an example, embodiments of the present invention can identify which of multiple factories produces deviant parameters (i.e., contaminated data), and assign a lower weight to the contaminated data or directly not use the contaminated data in the training of global parameters, so as to avoid training a model with too high a misjudgment rate and improve the accuracy of the model.

[0079] Although the present invention is disclosed in conjunction with the above embodiments, they are not intended to limit the present invention. Any ordinary technician in the technical field may make slight changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the definition of the attached claims.

Claims

1. A distributed training optimization method, comprising: Training the local model using one of the plurality of sample sets and the global parameters of the global model to generate local parameters of the local model; as well as At least one deviation parameter among the local parameters of the local models trained by the sample sets is determined, wherein the distribution of the at least one deviation parameter is far away from the distribution of other local parameters, and the local parameters of the local models are used to update the global parameters of the global model.

2. The distributed training optimization method of claim 1, wherein the local parameters of each of the local models include local correction parameters, the global parameters include global correction parameters, the global correction parameters are obtained by weighted calculation based on the local correction parameters, and the step of determining the at least one deviation parameter in the local parameters of the local models trained by the sample sets comprises: The at least one deviation parameter is determined according to the operation weight of the local correction parameter of each of the local models used in the weighted operation.

3. The distributed training optimization method of claim 2, wherein the step of determining the at least one deviation parameter according to the calculation weight used in the weighted calculation of the local correction parameter of each local model comprises: comparing the operation weight of the local correction parameter of each of the local models used in the weighted operation with a deviation threshold value; In response to the calculation weight of the local correction parameter of the local model used in the weighted calculation being less than the deviation threshold value, using the local correction parameter of the local model as the at least one deviation parameter; as well as In response to the fact that the calculation weight of the local correction parameter of the local model used in the weighted calculation is not less than the deviation threshold value, the local correction parameter of the local model is prohibited from being used as the at least one deviation parameter.

4. The distributed training optimization method according to claim 3, further comprising: Determining probability distribution of computational weights of the local modification parameters of the local models used in the weighted computation; as well as The value corresponding to the deviation quantile in the probability distribution is used as the deviation threshold value.

5. The distributed training optimization method as claimed in claim 4, wherein the probability distribution is a t distribution. 6 . The distributed training optimization method as claimed in claim 2 , wherein the local correction parameter is a gradient value determined based on a gradient descent method.

7. The distributed training optimization method of claim 1 , wherein the local parameters of each of the local models include local correction parameters, the global parameters include global correction parameters, and the step of determining the at least one deviation parameter in the local parameters of the local models trained by the sample sets comprises: The error detection model is used to determine whether the local correction parameter of each local model is the at least one deviation parameter.

8. The distributed training optimization method as claimed in claim 7, wherein the error detection model is obtained by training an autoencoder using a plurality of normal correction parameters.

9. The distributed training optimization method according to claim 1, further comprising: The at least one deviating parameter is alerted.

10. The distributed training optimization method according to claim 1, further comprising: Deleting the at least one deviation parameter from the local parameters of the local models trained by the sample sets; as well as Only other local parameters that are not the at least one deviation parameter are used to update the global parameters of the global model.

11. A main computing device, comprising: Memory, storing program codes; as well as A processor is coupled to the memory and loaded with the program code to execute: Obtaining local parameters corresponding to a plurality of sample sets, wherein the local parameters corresponding to each of the sample sets are generated by training a local model using one of the sample sets and a global parameter of a global model; as well as At least one deviation parameter among the local parameters of the local models trained by the sample sets is determined, wherein the distribution of the at least one deviation parameter is far away from the distribution of other local parameters, and the local parameters of the local models are used to update the global parameters of the global model.

12. The main computing device of claim 11, wherein the local parameters of each of the local models include local correction parameters, the global parameters include global correction parameters, the global correction parameters are obtained by weighted calculation based on the local correction parameters, and the processor further executes: The at least one deviation parameter is determined according to the operation weight of the local correction parameter of each of the local models used in the weighted operation.

13. The main computing device as claimed in claim 12, wherein the processor further executes: comparing the operation weight of the local correction parameter of each of the local models used in the weighted operation with a deviation threshold value; In response to the calculation weight of the local correction parameter of the local model used in the weighted calculation being less than the deviation threshold value, using the local correction parameter of the local model as the at least one deviation parameter; and In response to the fact that the calculation weight of the local correction parameter of the local model used in the weighted calculation is not less than the deviation threshold value, the local correction parameter of the local model is prohibited from being used as the at least one deviation parameter.

14. The main computing device as claimed in claim 13, wherein the processor further executes: Determining the probability distribution of the computational weights of the local modification parameters of the local models used in the weighted computation; and The value corresponding to the deviation quantile in the probability distribution is used as the deviation threshold value. The main computing device as claimed in claim 14 , wherein the probability distribution is a t distribution. 16 . The main computing device as claimed in claim 12 , wherein the local correction parameter is a gradient value determined based on a gradient descent method.

17. The main computing device of claim 11, wherein the local parameters of each of the local models include local correction parameters, the global parameters include global correction parameters, the global correction parameters are obtained by weighted calculation based on the local correction parameters, and the processor further executes: The error detection model is used to determine whether the local correction parameter of each local model is the at least one deviation parameter.

18. The main computing device as claimed in claim 17, wherein the error detection model is obtained by training an automatic encoder using a plurality of normal correction parameters.

19. The host computing device as claimed in claim 11, wherein the processor further executes: The at least one deviating parameter is alerted.

20. The host computing device as claimed in claim 11, wherein the processor further executes: Deleting the at least one deviation parameter from the local parameters of the local models trained by the sample sets; and Only other local parameters that are not the at least one deviation parameter are used to update the global parameters of the global model.