Distributed user-participated large model training system and method
By establishing a large model training system with distributed user participation between the data center and the user end, using the computing resources of the user equipment for local model training, and returning the results to the data center for merge and update, the problem of low efficiency of decentralized large model training in the existing technology is solved, and efficient and economical training effects are achieved.
Patent Information
- Application Number
- CN202510135937.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-07
- Publication Date
- 2025-06-06
AI Technical Summary
The prior art is difficult to perform large-scale training without adding additional costs and improve training efficiency, especially when utilizing distributed computing resources.
By establishing a large model training system with distributed user participation between the data center and the user side, the data center divides the training data into multiple training subsets and trains local models based on the computing resources of the user equipment. The user side downloads task packages for training when the device is idle, and returns the results to the data center for merge and update.
It realizes efficient use of distributed computing resources for large-scale training without adding additional costs, which significantly improves training efficiency, reduces hardware costs, and improves model performance and generalization capabilities.
Smart Images
Figure CN120106212A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a large model training system and method involving distributed users. Background Art
[0002] In the field of artificial intelligence and machine learning, the training of deep learning models, especially large neural network models, has become the key to promoting technological progress. However, the training of these large models usually requires huge computing resources and time. Traditional training methods mainly rely on high-performance computing (HPC) clusters or large data centers, which are not only costly but also often face the problem of insufficient resources.
[0003] With the rapid development of cloud computing, edge computing, and the Internet of Things, more and more devices are connected to the Internet, and these devices have different computing capabilities and storage resources. In this case, how to effectively utilize these decentralized and heterogeneous computing resources has become a challenging problem.
[0004] Although existing distributed computing frameworks such as Apache Hadoop and Apache Spark can achieve large-scale data processing, they often cannot fully utilize the performance of dedicated accelerators such as GPUs when processing deep learning model training, nor can they adapt well to the specific needs of deep learning model training.
[0005] In addition, traditional deep learning model training methods usually concentrate the entire training data set on a single or multiple high-performance servers for processing. This method not only requires a large amount of network bandwidth to transmit data, but also easily causes network congestion within the data center, affecting training efficiency.
[0006] How to conduct model training in a decentralized manner and improve training efficiency without incurring additional costs is a technical problem that needs to be solved. Summary of the invention
[0007] The technical task of the present invention is to address the above shortcomings and provide a large model training system and method with distributed user participation to solve the technical problem of how to conduct model training in a decentralized manner and improve training efficiency without increasing additional costs.
[0008] In a first aspect, the present invention provides a large model training system with distributed user participation, including a data center and multiple user terminals, wherein the user terminals include user devices and clients configured on the user devices;
[0009] The data center configures a training set and an initialized large model, and the data center is used to divide the training set into multiple training subsets, build a task package based on the training subsets, the initial weights of the large model and preset training parameters, and publish the task package;
[0010] The user terminal is used to select an adapted task package from the data center through the client based on the computing resources of its user device, download the task package when the user device is in an idle state, and perform local model training based on the training subset in the task package, the initial weight of the large model and the preset training parameters to obtain the weight of the large model, and return the weight of the large model as the training result to the data center;
[0011] Correspondingly, the data center user receives the training results uploaded by each user terminal, and merges the weights of the big model returned by each user terminal, updates the weight of the big model based on the merged result, performs model verification and model testing on the updated big model based on the training set to evaluate the model performance of the big model, and applies the updated big model to actual tasks or scenarios.
[0012] Preferably, the data center is also used to perform iterative optimization based on the training results returned by each user terminal and the model performance of the large model, and the iterative optimization is performed as follows: adjusting model training parameters, modifying model structure and adding training data, dividing the updated training set into multiple training subsets, constructing a task package based on the new training subsets, the updated weights of the large model and the modified training parameters, and publishing the task package;
[0013] Correspondingly, the user end is used to select an adapted task package from the data center through the client based on the computing resources of its user device, and download the task package when the user device is in an idle state, and perform local model training based on the new training subset in the task package, the updated weight of the large model and the modified training parameters to obtain the weight of the large model, and return the weight of the large model as the training result to the data center;
[0014] Correspondingly, the data center receives the training results uploaded by each user terminal, merges the weights of the big model returned by each user terminal, updates the weight of the big model based on the merged result, and performs model verification and model testing on the updated big model based on the training set to evaluate the model performance of the big model.
[0015] Preferably, the user terminal is also used to record the local model training process to form a local model training log, and perform model evaluation on the large model obtained through local model training to obtain performance indicators, and return the weight of the large model obtained through local model training, the local model training log and performance indicators to the data center as training results.
[0016] Preferably, the data center is used to perform data preprocessing and formatting operations on the training set to obtain a processed training set, and divide the processed training set into multiple training subsets, wherein outliers, erroneous data and duplicates in the training set are removed by data preprocessing, and the training set is converted into a data format suitable for the user end by formatting operations.
[0017] Preferably, the data center constructs a task package based on the training subset, the initial weight of the large model and preset training parameters, and compresses the task package;
[0018] Correspondingly, the user end downloads the compressed task package from the data center when its user device is in an idle state, and decompresses it to obtain the training subset, the initial weight of the large model, and the preset training parameters;
[0019] Correspondingly, the user end compresses its training results and returns them to the data center, and the data center receives the compressed training results and decompresses them.
[0020] In a second aspect, the present invention provides a large model training method with distributed user participation, which is used to perform large model training using a large model training system with distributed user participation as described in any one of the first aspects, and the method comprises the following steps:
[0021] Task allocation: Configure the training set and the initialized large model in the data center, divide the training set into multiple training subsets through the data center, build a task package based on the training subsets, the initial weights of the large model, and the preset training parameters, and publish the task package;
[0022] Local task execution: The user end selects an adapted task package from the data center through the client based on the computing resources of its user device, downloads the task package when the user device is in idle state, and performs local model training based on the training subset in the task package, the initial weight of the large model and the preset training parameters to obtain the weight of the large model, and returns the weight of the large model as the training result to the data center;
[0023] Model update: The data center user receives the training results uploaded by each user end, merges the weights of the big model returned by each user end, updates the weight of the big model based on the merged result, verifies and tests the updated big model based on the training set to evaluate the model performance of the big model, and applies the updated big model to actual tasks or scenarios.
[0024] As an advantage, the method further comprises the following steps:
[0025] Iterative optimization: The data center performs iterative optimization based on the training results returned by each user end and the model performance of the large model. The iterative optimization is performed as follows: adjusting model training parameters, modifying model structure, and adding training data, dividing the updated training set into multiple training subsets, and building a task package based on the new training subsets, the updated weights of the large model, and the modified training parameters, and publishing the task package;
[0026] Local task execution: The user end selects an adapted task package from the data center through the client based on the computing resources of its user device, and downloads the task package when the user device is in idle state. It then performs local model training based on the new training subset in the task package, the updated weight of the large model, and the modified training parameters to obtain the weight of the large model, and returns the weight of the large model as the training result to the data center;
[0027] Model update: The data center receives the training results uploaded by each user terminal, merges the weights of the big model returned by each user terminal, updates the weight of the big model based on the merged results, and performs model verification and model testing on the updated big model based on the training set to evaluate the model performance of the big model.
[0028] Preferably, when the local task is executed, the user end records the local model training process to form a local model training log, and performs model evaluation on the large model obtained through the local model training to obtain performance indicators, and returns the weight of the large model obtained through the local model training, the local model training log and the performance indicators to the data center as training results.
[0029] Preferably, during task allocation and iterative optimization, the data center performs data preprocessing and formatting operations on the training set to obtain a processed training set, and divides the processed training set into multiple training subsets, wherein data preprocessing is used to remove outliers, erroneous data, and duplicates in the training set, and formatting operations are used to convert the training set into a data format suitable for the user end.
[0030] As a preferred method, during task allocation and iterative optimization, the data center constructs a task package based on the training subset, the initial weight of the large model and the preset training parameters, and compresses the task package;
[0031] When training the local model, the user end downloads the compressed task package from the data center when its user device is in idle state, and decompresses it to obtain the training subset, the initial weight of the large model, and the preset training parameters;
[0032] The user end compresses its training results and returns them to the data center, which receives and decompresses the compressed training results.
[0033] The large model training system and method with distributed user participation of the present invention have the following advantages:
[0034] 1. Efficiency: Distributed training can make full use of the idle computing resources of online users and distribute the training tasks of large models to multiple user devices for parallel execution, thereby significantly improving training efficiency;
[0035] 2. It reduces the training bottleneck caused by data center resource limitations in traditional methods, allowing large models to be trained in a shorter time;
[0036] 3. Economical: No need to purchase or rent expensive high-performance computing clusters, reducing hardware costs. It fully utilizes the user's idle computing resources, reduces the waste of computing resources, and thus reduces the overall training cost. Due to the reduction in computing resource investment, it also reduces energy consumption and related environmental costs accordingly;
[0037] 4. Flexibility: Users can choose to participate in training according to their own equipment and network conditions. There is no need for special hardware requirements. Users can train when the equipment is idle without affecting the normal use of the user. The system can dynamically adjust task allocation according to the user's computing power and network conditions to achieve more efficient resource utilization;
[0038] 5. Scalability: As the number of users participating in training increases, the overall computing power of the system can grow linearly to meet the training needs of larger models. The distributed architecture enables the system to be easily expanded to cope with growing computing needs.
[0039] 6. Fault tolerance: In a distributed environment, even if some user devices or networks fail, other user devices can still continue training, ensuring the stability and reliability of training. The data center can verify and validate the training results uploaded by users to ensure the accuracy and consistency of the data;
[0040] 7. Data privacy protection: By splitting the training data into multiple task packages and assigning them to different users, the amount of sensitive data stored on a single user's device can be reduced, reducing the risk of data leakage. Distributed training can be performed without transmitting the original data, further protecting user data privacy;
[0041] 8. Real-time: Since user devices can participate in training online, the system can obtain and update the user's training results in real time, thereby adjusting and optimizing the model more quickly. Real-time enables the system to adapt to new data and task requirements more quickly, improving the practical application capabilities of the model;
[0042] 9. Sustainability: It reduces the demand for high-performance computing equipment and energy consumption, helps reduce the impact on the environment, achieves green computing, and promotes the rational use of resources and sustainable development by utilizing users' idle computing resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0044] The present invention is further described below in conjunction with the accompanying drawings.
[0045] Figure 1 This is a schematic diagram of distributed training in a large model training system with distributed user participation in Example 1;
[0046] Figure 2 This is a flowchart of a large model training method involving distributed users in Example 2. DETAILED DESCRIPTION
[0047] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.
[0048] The embodiments of the present invention provide a large model training system and method with distributed user participation, which are used to solve the technical problem of how to conduct model training in a decentralized manner and improve training efficiency without increasing additional costs.
[0049] Embodiment 1:
[0050] The present invention discloses a large model training system with distributed user participation, comprising a data center and multiple user terminals, wherein the user terminals comprise user equipment and clients configured on the user equipment.
[0051] The training set and the initialized large model are configured in the data center. The data center is used to divide the training set into multiple training subsets, build a task package based on the training subsets, the initial weights of the large model and the preset training parameters, and publish the task package.
[0052] As a specific implementation of the data center, the data center is used to perform data preprocessing and formatting operations on the training set, obtain the processed training set, and divide the processed training set into multiple training subsets, wherein the outliers, erroneous data and duplicates in the training set are removed through data preprocessing, and the training set is converted into a data format suitable for the user end, such as tensor, etc. through formatting operations.
[0053] In this embodiment, the data center loads the large model to be trained and the training data set, and preprocesses and formats the training data set. The role of preprocessing and formatting is to improve the quality of the data, identify and correct errors, anomalies or duplicates in the data set, thereby improving the accuracy, consistency and reliability of the data. According to certain rules, the data is cut into independent block tasks, such as by storage size, or by the integrity and independence of the task, for users to choose to download block tasks. The data center classifies different tasks and publishes them online, and users download the tasks to their local computer and then conduct training.
[0054] In this embodiment, the task package involves a training subset, initial weights of a large model, a training script (such as a Python script), training parameters (such as a learning rate, batch size, etc.), and any necessary dependent libraries or data preprocessing codes.
[0055] The user end is used to select an adapted task package from the data center through the client based on the computing resources of its user device, download the task package when the user device is in idle state, and perform local model training based on the training subset in the task package, the initial weight of the large model and the preset training parameters to obtain the weight of the large model, and return the weight of the large model to the data center as the training result.
[0056] As a specific implementation of the user end, the user end is also used to record the local model training process to form a local model training log, and perform model evaluation on the large model obtained through local model training to obtain performance indicators, and return the weight of the large model obtained through local model training, the local model training log and performance indicators as training results to the data center.
[0057] In this embodiment, the user device is a terminal that performs training tasks, which can be a desktop, laptop, GPU server, etc. The user can download the task package from the data center to the local through the user device, and perform the training task when the device is idle. The user device needs to have certain computing power and storage capacity to support the training of large models. At the same time, the user device also needs to support stable and efficient data transmission with the data center. The user device downloads the task package from the server in the data center based on a secure network connection such as HTTPS, which requires the user device to have sufficient storage space and appropriate network bandwidth.
[0058] During training, it is necessary to detect the idle state of the software on the user's device (such as mobile applications, desktop applications, or background services), which is usually based on the CPU, GPU, and memory usage, etc. Task training can only be performed when the user allows it or the software is in an idle state.
[0059] When the user allows or the device is idle, the client executes the downloaded training task from the data center, which may include loading the model, loading the data, performing forward and backward propagation, and updating the model weights.
[0060] Since user devices are usually not designed for sustained high-intensity computing, training tasks may be affected by resource limitations (such as battery power, heat dissipation capacity, network bandwidth, etc.).
[0061] Correspondingly, the data center user receives the training results uploaded by each user terminal, and merges the weights of the big model returned by each user terminal, updates the weight of the big model based on the merged result, performs model verification and model testing on the updated big model based on the training set to evaluate the model performance of the big model, and applies the updated big model to actual tasks or scenarios.
[0062] In this embodiment, the data center is used to aggregate the training results from multiple user devices and merge them according to the task cutting order to further improve the performance of the model or train a larger-scale model, and apply the updated model to actual tasks or scenarios.
[0063] Among them, the data center builds a task package based on the training subset, the initial weights of the large model and the preset training parameters, and compresses the task package; correspondingly, the user end downloads the compressed task package from the data center when its user device is in idle state, and decompresses it to obtain the training subset, the initial weights of the large model and the preset training parameters; correspondingly, the user end compresses its training results and returns them to the data center, and the data center receives the compressed training results and decompresses them.
[0064] As an improvement of the system of this embodiment, the data center is also used to perform iterative optimization based on the training results returned by each user terminal and the model performance of the large model. The iterative optimization is performed as follows: adjusting model training parameters, modifying model structure, and adding training data, dividing the updated training set into multiple training subsets, building a task package based on the new training subsets, the updated weights of the large model and the modified training parameters, and publishing the task package.
[0065] Correspondingly, the user end is used to select an adapted task package from the data center through the client based on the computing resources of its user device, and download the task package when the user device is in idle state, and perform local model training based on the new training subset in the task package, the updated weight of the large model and the modified training parameters to obtain the weight of the large model, and return the weight of the large model to the data center as the training result.
[0066] Correspondingly, the data center receives the training results uploaded by each user terminal, merges the weights of the big model returned by each user terminal, updates the weight of the big model based on the merged result, and performs model verification and model testing on the updated big model based on the training set to evaluate the model performance of the big model.
[0067] Through the above-mentioned improved implementation example, the data center will perform iterative optimization based on the user's training results and model performance, including adjusting the learning rate, modifying the model structure, adding training data, and other operations. Through continuous iterative optimization, the performance and generalization ability of the large model can be gradually improved.
[0068] Embodiment 2:
[0069] The present invention discloses a large model training method with distributed user participation, including task allocation, local task execution and model update.
[0070] Step S100: Task allocation: configure a training set and an initialized large model in a data center, divide the training set into multiple training subsets through the data center, build a task package based on the training subsets, the initial weights of the large model and preset training parameters, and publish the task package.
[0071] The data center is used to perform data preprocessing and formatting operations on the training set, obtain the processed training set, and divide the processed training set into multiple training subsets. Among them, data preprocessing is used to remove outliers, erroneous data, and duplicates in the training set, and formatting operations are used to convert the training set into a data format suitable for the user end, such as tensor.
[0072] In this embodiment, the data center loads the large model to be trained and the training data set, and preprocesses and formats the training data set. The role of preprocessing and formatting is to improve the quality of the data, identify and correct errors, anomalies or duplicates in the data set, thereby improving the accuracy, consistency and reliability of the data. According to certain rules, the data is cut into independent block tasks, such as by storage size, or by the integrity and independence of the task, for users to choose to download block tasks. The data center classifies different tasks and publishes them online, and users download the tasks to their local computer and then conduct training.
[0073] The task package involves the training subset, the initial weights of the large model, the training script (such as a Python script), the training parameters (such as learning rate, batch size, etc.), and any necessary dependent libraries or data preprocessing codes.
[0074] Step S200 local task execution: the user end selects an adapted task package from the data center through the client based on the computing resources of its user device, downloads the task package when the user device is in idle state, and performs local model training based on the training subset in the task package, the initial weight of the large model and the preset training parameters to obtain the weight of the large model, and returns the weight of the large model to the data center as the training result.
[0075] As a specific implementation of local task execution, when the user side performs local model training, it records the local model training process to form a local model training log, and performs model evaluation on the large model obtained through local model training to obtain performance indicators. The weight of the large model obtained through local model training, the local model training log and performance indicators are returned to the data center as training results.
[0076] In this embodiment, the user device is a terminal that performs training tasks, which can be a desktop, laptop, GPU server, etc. The user can download the task package from the data center to the local through the user device, and perform the training task when the device is idle. The user device needs to have certain computing power and storage capacity to support the training of large models. At the same time, the user device also needs to support stable and efficient data transmission with the data center. The user device downloads the task package from the server in the data center based on a secure network connection such as HTTPS, which requires the user device to have sufficient storage space and appropriate network bandwidth.
[0077] During training, it is necessary to detect the idle state of the software on the user's device (such as mobile applications, desktop applications, or background services), which is usually based on the CPU, GPU, and memory usage, etc. Task training can only be performed when the user allows it or the software is in an idle state.
[0078] When the user allows or the device is idle, the client executes the downloaded training task from the data center, which may include loading the model, loading the data, performing forward and backward propagation, and updating the model weights.
[0079] Since user devices are usually not designed for sustained high-intensity computing, training tasks may be affected by resource limitations (such as battery power, heat dissipation capacity, network bandwidth, etc.).
[0080] Step S300 model update: The data center user receives the training results uploaded by each user terminal, and merges the weights of the big model returned by each user terminal, updates the weight of the big model based on the merged result, performs model verification and model testing on the updated big model based on the training set to evaluate the model performance of the big model, and applies the updated big model to actual tasks or scenarios.
[0081] In this embodiment, the data center is used to aggregate the training results from multiple user devices and merge them according to the task cutting order to further improve the performance of the model or train a larger-scale model, and apply the updated model to actual tasks or scenarios.
[0082] Among them, the data center builds a task package based on the training subset, the initial weights of the large model and the preset training parameters, and compresses the task package; correspondingly, the user end downloads the compressed task package from the data center when its user device is in idle state, and decompresses it to obtain the training subset, the initial weights of the large model and the preset training parameters; correspondingly, the user end compresses its training results and returns them to the data center, and the data center receives the compressed training results and decompresses them.
[0083] As an improvement to the method of this embodiment, the following steps are also included:
[0084] Step S400 iterative optimization: The data center performs iterative optimization based on the training results returned by each user terminal and the model performance of the large model. The iterative optimization is performed as follows: adjusting model training parameters, modifying model structure, and adding training data, dividing the updated training set into multiple training subsets, and constructing a task package based on the new training subsets, the updated weights of the large model, and the modified training parameters, and publishing the task package.
[0085] Step S500 local model training: The user end is used to select an adapted task package from the data center through the client based on the computing resources of its user device, and download the task package when the user device is in an idle state, and perform local model training based on the new training subset in the task package, the updated weight of the large model and the modified training parameters to obtain the weight of the large model, and return the weight of the large model to the data center as the training result.
[0086] Step S600: Model update: The data center receives the training results uploaded by each user terminal, and merges the weights of the large model returned by each user terminal, updates the weight of the large model based on the merged result, and performs model verification and model testing on the updated large model based on the training set to evaluate the model performance of the large model.
[0087] In specific implementation, the data center cooperates with the user end to cyclically execute steps S400 to S600 to iteratively optimize the large model. Through continuous iterative optimization, the performance and generalization ability of the large model can be gradually improved.
[0088] The above is a detailed introduction to the large-scale model training system and method with distributed user participation provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A large model training system with distributed user participation, characterized in that: It includes a data center and multiple user terminals, wherein the user terminals include user equipment and clients configured on the user equipment; The data center configures a training set and an initialized large model, and the data center is used to divide the training set into multiple training subsets, build a task package based on the training subsets, the initial weights of the large model and preset training parameters, and publish the task package; The user terminal is used to select an adapted task package from the data center through the client based on the computing resources of its user device, download the task package when the user device is in an idle state, and perform local model training based on the training subset in the task package, the initial weight of the large model and the preset training parameters to obtain the weight of the large model, and return the weight of the large model as the training result to the data center; Correspondingly, the data center user receives the training results uploaded by each user terminal, and merges the weights of the big model returned by each user terminal, updates the weight of the big model based on the merged result, performs model verification and model testing on the updated big model based on the training set to evaluate the model performance of the big model, and applies the updated big model to actual tasks or scenarios.
2. The large model training system with distributed user participation according to claim 1 is characterized in that: The data center is also used to perform iterative optimization based on the training results returned by each user terminal and the model performance of the large model. The iterative optimization is performed as follows: adjusting model training parameters, modifying model structure and adding training data, dividing the updated training set into multiple training subsets, building a task package based on the new training subsets, the updated weights of the large model and the modified training parameters, and publishing the task package; Correspondingly, the user end is used to select an adapted task package from the data center through the client based on the computing resources of its user device, and download the task package when the user device is in an idle state, and perform local model training based on the new training subset in the task package, the updated weight of the large model and the modified training parameters to obtain the weight of the large model, and return the weight of the large model as the training result to the data center; Correspondingly, the data center receives the training results uploaded by each user terminal, merges the weights of the big model returned by each user terminal, updates the weight of the big model based on the merged result, and performs model verification and model testing on the updated big model based on the training set to evaluate the model performance of the big model.
3. The large model training system with distributed user participation according to claim 1 or 2, characterized in that: The user terminal is also used to record the local model training process to form a local model training log, and perform model evaluation on the large model obtained through local model training to obtain performance indicators, and return the weight of the large model obtained through local model training, the local model training log and performance indicators as training results to the data center.
4. The large model training system with distributed user participation according to claim 1 or 2, characterized in that: The data center is used to perform data preprocessing and formatting operations on the training set to obtain a processed training set, and divide the processed training set into multiple training subsets, wherein outliers, erroneous data and duplicates in the training set are removed through data preprocessing, and the training set is converted into a data format suitable for the user end through formatting operations.
5. The large model training system with distributed user participation according to claim 1 or 2, characterized in that: The data center constructs a task package based on the training subset, the initial weight of the large model and the preset training parameters, and compresses the task package; Correspondingly, the user end downloads the compressed task package from the data center when its user device is in an idle state, and decompresses it to obtain the training subset, the initial weight of the large model, and the preset training parameters; Correspondingly, the user end compresses its training results and returns them to the data center, and the data center receives the compressed training results and decompresses them.
6. A large model training method with distributed user participation, characterized in that: The method is used for performing large model training through a large model training system with distributed user participation as described in any one of claims 1 to 5, and the method comprises the following steps: Task allocation: Configure the training set and the initialized large model in the data center, divide the training set into multiple training subsets through the data center, build a task package based on the training subsets, the initial weights of the large model, and the preset training parameters, and publish the task package; Local task execution: The user end selects an adapted task package from the data center through the client based on the computing resources of its user device, downloads the task package when the user device is in idle state, and performs local model training based on the training subset in the task package, the initial weight of the large model and the preset training parameters to obtain the weight of the large model, and returns the weight of the large model as the training result to the data center; Model update: The data center user receives the training results uploaded by each user end, merges the weights of the big model returned by each user end, updates the weight of the big model based on the merged result, verifies and tests the updated big model based on the training set to evaluate the model performance of the big model, and applies the updated big model to actual tasks or scenarios.
7. The large model training method with distributed user participation according to claim 6 is characterized in that: The following steps are also included: Iterative optimization: The data center performs iterative optimization based on the training results returned by each user end and the model performance of the large model. The iterative optimization is performed as follows: adjusting model training parameters, modifying model structure, and adding training data, dividing the updated training set into multiple training subsets, and building a task package based on the new training subsets, the updated weights of the large model, and the modified training parameters, and publishing the task package; Local task execution: The user end selects an adapted task package from the data center through the client based on the computing resources of its user device, and downloads the task package when the user device is in idle state. It then performs local model training based on the new training subset in the task package, the updated weight of the large model, and the modified training parameters to obtain the weight of the large model, and returns the weight of the large model as the training result to the data center; Model update: The data center receives the training results uploaded by each user terminal, merges the weights of the big model returned by each user terminal, updates the weight of the big model based on the merged results, and performs model verification and model testing on the updated big model based on the training set to evaluate the model performance of the big model.
8. The large model training method with distributed user participation according to claim 6 or 7, characterized in that: When the local task is executed, the user end records the local model training process to form a local model training log, and performs model evaluation on the large model obtained through local model training to obtain performance indicators. The weight of the large model obtained through local model training, the local model training log and performance indicators are returned to the data center as training results.
9. The large model training method with distributed user participation according to claim 7 is characterized in that: During task allocation and iterative optimization, the data center performs data preprocessing and formatting operations on the training set to obtain a processed training set, and divides the processed training set into multiple training subsets. Data preprocessing is used to remove outliers, erroneous data, and duplicates in the training set, and formatting operations are used to convert the training set into a data format that is suitable for the user end.
10. The large model training method with distributed user participation according to claim 7, characterized in that: During task allocation and iterative optimization, the data center builds task packages based on the training subset, the initial weights of the large model, and the preset training parameters, and compresses the task packages; When training the local model, the user end downloads the compressed task package from the data center when its user device is in idle state, and decompresses it to obtain the training subset, the initial weight of the large model, and the preset training parameters; The user end compresses its training results and returns them to the data center, which receives and decompresses the compressed training results.