Federal instruction tuning method and system based on data quality control

By using instruction-response consistency scoring and hierarchical training methods in federated learning, the challenge of data quality control of large language models in federated learning environment is solved, the robustness and training efficiency of the model are improved, and data privacy is ensured. Protect.

CN119026704AActive Publication Date: 2024-11-26SHANGHAI JIAOTONG UNIV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411116906.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-11-26
Estimated Expiration
2044-08-15

AI Technical Summary

Technical Problem

Large language models face the challenge of data quality control in a federated learning environment. The lack of effective data quality control methods in the existing technology leads to inefficient training and slowing down convergence speed, especially when facing complex samples in the initial stage.

Method used

Send global initial model parameters and quality thresholds to the client through the server, use the instruction-response consistency score to evaluate the data quality, filter low-quality data, and sort high-quality data by hierarchy. The client uses data samples to train locally in turn. The server Update global model parameters by weighted average, repeating this process until the preset training round is reached.

Benefits of technology

A globally consistent and efficient data quality evaluation is achieved, which improves the robustness of the model to noise, improves training effect and performance, reduces the negative impact of low-quality data, and prevents model overfitting.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119026704A_ABST
    Figure CN119026704A_ABST
Patent Text Reader

Abstract

The invention provides a federal instruction tuning method and system based on data quality control, and the method comprises the steps: carrying out the instruction-response consistency scoring of local data of each client through employing a global initial model parameter, and evaluating the quality of each sample; filtering low-quality data according to a preset quality threshold value, and retaining data samples with consistency scores higher than the threshold value; the filtered sample data are sorted from high to low according to consistency scores, the sample data are divided into a plurality of levels, and each level comprises the same number of data samples; each client carries out local training by using the data samples in sequence according to a hierarchical sequence; after each round of training is finished, the server receives local updating parameters of all participating clients, and global model parameters are updated in a weighted average mode; and repeating the steps until a preset round is reached. According to the method, global consistent and efficient data quality evaluation is implemented, and a hierarchical training method is combined, so that the robustness of the model to noise can be improved, and the training effect and performance of the model are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a federal instruction tuning method and system based on data quality control. Background Art

[0002] With the development of big data and artificial intelligence, the performance of machine learning models is increasingly dependent on the quantity and quality of training data. However, in many privacy-sensitive fields, such as medicine and finance, data privacy issues limit the centralized sharing and utilization of large-scale data. Federated learning, as a distributed training method, solves the data privacy problem by allowing multiple clients to collaboratively train a unified model without sharing their data, and has broad application prospects. Large language models are a type of model that has made significant progress in the field of natural language processing in recent years. Through large-scale instruction-response data training, the model can better understand and perform complex tasks.

[0003] However, instruction tuning of large language models in a federated learning environment faces the challenge of data quality control. Due to the diverse data sources of each client and the uneven data quality, it is difficult to detect and filter low-quality data, which directly affects the training effect and final performance of the model. Current data quality control schemes for large language models focus on evaluating data quality. During the training process, if related issues, especially the importance of data quality, are ignored, it will lead to low training efficiency and a significant slowdown in convergence speed. If the model randomly encounters data of different difficulty levels during training, especially when faced with a large number of complex samples in the initial stage, this situation is very likely to lead to increased instability in the learning process, and thus negatively affect the training convergence speed.

[0004] Existing federated methods mainly focus on data volume expansion and privacy protection, but lack in-depth research on data quality control. Current work on data quality in federated learning is mainly concentrated in classification tasks, most of which are difficult to transfer to language model generation tasks and difficult to adapt to real scenarios in federated learning. Summary of the invention

[0005] In view of the defects in the prior art, the object of the present invention is to provide a federal instruction tuning method and system based on data quality control.

[0006] According to one aspect of the present invention, a method for tuning a federal instruction based on data quality control is provided, comprising:

[0007] The server sends global initial model parameters and quality thresholds to the client;

[0008] Using the global initial model parameters, the local data samples of each client are scored for command-response consistency to evaluate the quality of each data sample;

[0009] Filter low-quality data according to the quality threshold, and only retain data samples whose command-response consistency score is higher than the quality threshold;

[0010] sorting the filtered sample data from high to low according to the command-response consistency score, and dividing the data into multiple levels, each level containing the same number of data samples;

[0011] Each client uses data samples for local training in descending order of hierarchy;

[0012] After each round of training, the server receives the local updated parameters of all participating clients and updates the global model parameters by weighted averaging;

[0013] Repeat the above steps until the preset training rounds are reached.

[0014] Preferably, the use of the global initial model parameters to score the command-response consistency of the local data samples of each client to evaluate the quality of each data sample is specifically as follows:

[0015] f(q i , a i ,θ 0 )=L(a i θ 0 )-L((q i , a i );θ 0 )

[0016] Where f is the function for calculating data quality, i.e., high command-response consistency score; (q i , a i )∈D n , D n is the local client dataset of user n, n∈[1,N], N is the number of users participating in federated learning; L(a i θ 0 ) means that there is no instruction q i Generate response a in case i The loss, L((q i , a i );θ 0 ) indicates that in the given instruction q i Generate response a in case i The loss of θ 0 are the global initial model parameters.

[0017] Preferably, filtering low-quality data according to the quality threshold and retaining only data samples whose command-response consistency score is higher than the quality threshold is specifically as follows:

[0018] D′ n ={(q i , a i )∈D n |f(q i , a i ,θ 0 )≥λ}

[0019] Where D n is the local dataset of user n, D′ n is the filtered data set, and λ is the set quality threshold.

[0020] Preferably, the filtered sample data is sorted from high to low according to the command-response consistency score and divided into multiple levels, each level containing the same number of data samples, specifically:

[0021]

[0022] Among them (q i , a i ) is taken from the data set after sorting in ascending order m is the filtered local data set D′ n The amount of data, k represents the level, and n represents the client.

[0023] Preferably, each client uses data samples for local training in order of levels, and gradually introduces data samples of lower-scoring levels, specifically:

[0024] Client n selects the data H corresponding to the corresponding level k according to the training round nk Perform local training and update local model parameters

[0025] Assume that the total number of training rounds is R, except for the last layer, the number of training rounds for each layer is The number of training rounds for the last layer is

[0026] Preferably, updating the local model parameters is achieved by gradient descent:

[0027]

[0028] Where η is the learning rate, is the local loss function, For parameters The gradient, Initialize to t is the current number of local update steps, and the local update stops at t = T, and T is the number of steps in each round of local update.

[0029] Preferably, after each round of training, the server receives the local updated parameters of all participating clients and updates the global model parameters by weighted averaging, specifically:

[0030]

[0031] Where |D n | is the data set D n The amount of data, The weights are usually assigned according to the size of each client’s dataset, S r is the set of users participating in this round of training. If all users participate in this round of training, then S r ={1,2,…,N}.

[0032] According to a second aspect of the present invention, a federal instruction tuning system based on data quality control is provided, comprising:

[0033] Initialization module: The server sends global initial model parameters and quality thresholds to the client;

[0034] Data quality assessment module: using the global initial model parameters to score the instruction-response consistency of each client's local data sample, and assessing the quality of each data sample;

[0035] Data filtering module: filtering low-quality data according to the quality threshold, and only retaining data samples with the instruction-response consistency score higher than the quality threshold;

[0036] Data sorting and stratification module: sorts the filtered sample data from high to low according to the command-response consistency score and divides it into multiple levels, each of which contains the same number of data samples;

[0037] Hierarchical training module: Each client uses data samples in order of hierarchy for local training, and gradually introduces data samples of lower-scoring hierarchies;

[0038] Global model aggregation module: After each round of training, the server receives the local updated parameters of all participating clients and updates the global model parameters by weighted averaging;

[0039] Repeating module: Repeat the data quality assessment module, data filtering module, data sorting and stratification module, stratification training module and global model aggregation module until the preset training rounds are reached.

[0040] According to a third aspect of the present invention, there is provided a terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can be used to execute the method described, or to run the system described, when executing the program.

[0041] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to execute the method described, or, to execute the system described.

[0042] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:

[0043] A federal instruction tuning method based on data quality control in an embodiment of the present invention implements globally consistent and efficient data quality assessment, and combined with a layered training method, can improve the robustness of the model to noise and enhance the model training effect and performance.

[0044] A federal instruction tuning method based on data quality control in an embodiment of the present invention can effectively reduce the negative impact of low-quality data: by filtering low-quality data, it ensures that the data involved in training has a high consistency, thereby improving the overall performance of the model.

[0045] A federal instruction tuning method based on data quality control in an embodiment of the present invention can improve the efficiency and robustness of model training: the quality-aware hierarchical training method enables the model to gradually adapt to the complexity of the data, prevents the model from overfitting to noisy data, and improves the robustness and training efficiency of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0047] Figure 1 It is a flow chart of a method for tuning federal instructions based on data quality control in a preferred embodiment of the present invention;

[0048] Figure 2 It is a schematic diagram of a federal instruction tuning method based on data quality control in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0049] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0050] The entire framework of the federated instruction tuning method based on data quality control in the embodiment of the present invention is based on the federated average method, which is the most basic method in federated learning. On this basis, data quality assessment and data training methods are added. The process of the federated average method is as follows:

[0051] The global model needs to be initialized before the federated training starts. That is, at the beginning of the training, the server initializes a global model parameter θ 0 , the model will be used in the subsequent federated training process.

[0052] In each training round:

[0053] First, client selection is required. The server randomly selects a portion of clients n∈S from all clients. r Participate in the training of the current round r. The purpose of this is to reduce communication overhead and computational burden.

[0054] Next, in local model training, each selected client n receives the current global model parameters from the server.

[0055] The client then uses its local dataset D n Perform local training updates on model parameters.

[0056] The training process usually minimizes the local loss function L through multiple local gradient descents (or other optimization methods). n (θ):

[0057]

[0058] Where η is the learning rate, is the local loss function, For parameters The gradient, Initialize to t is the current number of local update steps, and the local update stops at t = T, and T is the number of steps in each round of local update.

[0059] After each client completes local training, the updated model parameters Upload to the server. At this point, the server will receive updated model parameters from multiple clients. After each round, the server aggregates the global model. The server receives the local updated parameters of all participating clients and updates the global model parameters by weighted average:

[0060]

[0061] in The weights are usually assigned according to the size of each client’s dataset, S r is the set of users participating in this round of training. If all users participate in this round of training, then S r ={1,2,…,N},|D n | is the data set D n Repeat the above steps until the training round r=R.

[0062] The method for tuning the federal instruction based on data quality control proposed in the embodiment of the present invention has two innovative adjustments compared to the federated average method. The first is to add a data quality assessment and filtering part after initializing the global model and before training. The second is to divide the data into multiple subsets and perform layered training during the model training phase. Specifically, in one embodiment of the present invention, a method for tuning the federal instruction based on data quality control is provided, such as Figure 1 and Figure 2 As shown, the main process is as follows:

[0063] S1, the server sends the global initial model parameters and quality threshold to the client;

[0064] S2, using the global initial model parameters to score the command-response consistency of each client's local data and evaluate the quality of each data sample;

[0065] S3, filters low-quality data according to a preset quality threshold, and only retains data samples whose command-response consistency scores calculated by S2 are higher than the threshold;

[0066] S4, sorting the sample data after the filtering in S3 from high to low according to the command-response consistency score, and dividing it into multiple levels, each level containing the same number of data samples;

[0067] S5, each client uses the data samples for local training in the order of the levels divided in S4, and gradually introduces data samples of the lower-scoring levels;

[0068] S6, after each round of training, the server receives the local updated parameters of all participating clients and updates the global model parameters by weighted averaging;

[0069] Repeat the above steps S1-S6 until the preset training rounds are reached.

[0070] In the above embodiments, by implementing globally consistent and efficient data quality assessment during the training process and combining the layered training method, the robustness of the model to noise can be improved, and the model training effect and performance can be improved. This changes the situation in which the importance of data quality is ignored during the training process in the prior art.

[0071] In a preferred embodiment of the present invention, S2 is implemented to evaluate the quality of each data sample. In this embodiment, in view of the data quality problem, a data quality evaluation indicator is proposed as command-response consistency. This indicator evaluates data quality by calculating the difference between the unconditional response generation loss and the conditional response generation loss of each data sample.

[0072] Specifically, the instruction-response consistency score is calculated as follows: Use the global initial model to infer the local data of each client, and calculate the loss L(a) of each sample generating a response without instructions. i ; θ) and the loss L((q i ,a i );θ). The instruction-response consistency score is defined as the difference between the two, that is,

[0073] f(q i ,a i ,θ)=L(a i ;θ)-L((q i ,a i ),

[0074] Where f is a function that calculates data quality, which takes a data point and a model as input and outputs a vector representing the data quality of that data point. A high command-response consistency score indicates that the command q i and response a i A high correlation between the two scores indicates high data quality, while a low command-response consistency score indicates low data quality and may be noise data.

[0075] After completing the above data quality assessment, the data needs to be filtered. In a preferred embodiment of the present invention, S2 is implemented to provide a better filtering method. That is, each client filters the data samples according to the globally set quality threshold λ. The specific operation is to filter out samples below the threshold and only retain high-quality data. This process is completely performed locally on the client to ensure data privacy.

[0076] The formula representing the filtering process is:

[0077] D′ n ={(q i ,a i )∈D n |f(q i ,a i ,θ 0 )≥λ}

[0078] Where D n is the local dataset of client n, D′ nThe filtered data set.

[0079] In the above embodiment, low-quality data is filtered to ensure that the data involved in training has high consistency, thereby improving the overall performance of the model.

[0080] In a preferred embodiment of the present invention, S4 is implemented to sort the filtered data from high to low according to the command-response consistency score, and further divide it into multiple levels. Each level contains approximately the same number of data samples, aiming to help the model learn more effectively by gradually introducing complexity.

[0081] Among them, the formula for sorting and stratification is:

[0082]

[0083] Among them (q i ,a i ) is taken from the data set after sorting in ascending order m is the filtered local dataset D ′ n The amount of data.

[0084] In a preferred embodiment of the present invention, S5 is implemented to perform layer-by-layer training. The client uses the data in sequence according to the layer order to perform local training.

[0085] Use the highest rated tier first Train, then use the lower-scoring layers in turn Conduct training.

[0086] Assuming the total number of training rounds is R, then except for the last layer, the number of training rounds for each layer is The number of training rounds for the last layer is This design aims to ensure that each layer of data can be processed with the same number of steps as much as possible during the training process, so as to achieve a more balanced and efficient training effect.

[0087] In the above embodiment, a layered training strategy is adopted to ensure that the model can first build a solid foundation on high-quality, highly relevant data. After that, the model gradually adapts to a more complex data environment, effectively avoiding the overfitting problem and noise interference that may be caused by directly using the overall data. This training process not only significantly improves the training efficiency and effectiveness, but also significantly enhances the robustness of the model. In addition, all data processing (including evaluation, filtering and sorting) is performed locally on the client, and does not involve uploading and sharing of original data, so data privacy is fully guaranteed.

[0088] In a preferred embodiment of the present invention, S6 is implemented to perform parameter aggregation. Specifically, after each round of training, the server receives the local updated parameters of all participating clients and updates the global model parameters by weighted average, specifically:

[0089]

[0090] in The weights are usually assigned according to the size of each client's dataset. n | is the data set D n The amount of data, S r is the set of users participating in this round of training. If all users participate in this round of training, then S r ={1,2,…,N}.

[0091] The local update and global aggregation of each round are the same as those of the federated averaging method, and the entire training process is repeated until the training round r = R.

[0092] Based on the same inventive concept, in other embodiments of the present invention, a federal instruction tuning system based on data quality control is provided, including:

[0093] Initialization module: The server sends global initial model parameters and quality thresholds to the client;

[0094] Data quality assessment module: using the global initial model parameters to score the instruction-response consistency of each client's local data sample, and assessing the quality of each data sample;

[0095] Data filtering module: filtering low-quality data according to the quality threshold, and only retaining data samples with the instruction-response consistency score higher than the quality threshold;

[0096] Data sorting and stratification module: sorts the filtered sample data from high to low according to the command-response consistency score and divides it into multiple levels, each of which contains the same number of data samples;

[0097] Hierarchical training module: Each client uses data samples in order of hierarchy for local training, and gradually introduces data samples of lower-scoring hierarchies;

[0098] Global model aggregation module: After each round of training, the server receives the local updated parameters of all participating clients and updates the global model parameters by weighted averaging;

[0099] Repeating module: Repeat the data quality assessment module, data filtering module, data sorting and stratification module, stratification training module and global model aggregation module until the preset training rounds are reached.

[0100] The various modules / units in the above examples of the present invention may specifically refer to the implementation techniques of the corresponding steps of the federal instruction tuning method based on data quality control in the above embodiments, which will not be described in detail here.

[0101] Based on the same inventive concept, in other embodiments of the present invention, a terminal is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute the federal instruction tuning method based on data quality control, or to run the federal instruction tuning system based on data quality control.

[0102] Based on the same inventive concept, in other embodiments of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, it can be used to execute the federal instruction tuning method based on data quality control, or execute the federal instruction tuning system based on data quality control.

[0103] The technical solutions in the above embodiments of the present invention can be applied in many industrial application scenarios, especially in many real-world application scenarios, where federated instruction tuning methods based on data quality control are required. For example, in the medical field, federated learning can be used to analyze patient data from different hospitals or clinics to improve disease diagnosis, treatment plans, and drug discovery while protecting patient privacy.

[0104] For example, federated learning is needed to predict a patient's resistance to a certain therapeutic drug. The steps are as follows:

[0105] S100. In each medical institution participating in federated learning, use the global initial model to score the instruction-response consistency of each institution's local data (patient data samples) to evaluate the quality of each patient's data sample;

[0106] S200. In each medical institution, screen the patient data locally and only retain the data with a command-response consistency score higher than the set quality threshold;

[0107] S300, in each medical institution, the screened patient data is divided into levels, and the amount of data in each level remains the same;

[0108] S400, performing hierarchical training in each medical institution and updating the model parameters of each medical institution;

[0109] S500, aggregating the updated model parameters of each medical institution to update the global model.

[0110] S600: Repeat the federal instruction tuning method based on data quality control of S100-S500, iterate the model training, optimize the model parameters, and improve the model performance.

[0111] S700. Deploy the trained model to medical institutions with relevant application needs for actual medical diagnosis and treatment.

[0112] In the above process, all patient data processing (including evaluation, screening and sorting) is strictly performed locally on the client, without any uploading and sharing of original data, in order to ensure that data privacy is fully protected. At the same time, by implementing a globally unified and efficient data quality assessment mechanism, combined with a hierarchical training method, the model's tolerance to noise is effectively improved, thereby enhancing the accuracy of predicting patients' resistance to specific therapeutic drugs.

[0113] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various modifications or variations within the scope of the claims, which does not affect the essence of the present invention. The above preferred features can be used in any combination without conflicting with each other.

Claims

1. A method for tuning federal instructions based on data quality control, characterized in that: include: The server sends global initial model parameters and quality thresholds to the client; Using the global initial model parameters, the local data samples of each client are scored for command-response consistency to evaluate the quality of each data sample; Filter low-quality data according to the quality threshold, and only retain data samples whose command-response consistency score is higher than the quality threshold; sorting the filtered sample data from high to low according to the command-response consistency score, and dividing the data into multiple levels, each level containing the same number of data samples; Each client uses data samples for local training in descending order of hierarchy; After each round of training, the server receives the local updated parameters of all participating clients and updates the global model parameters by weighted averaging; Repeat the above steps until the preset training rounds are reached.

2. The method for tuning federal instructions based on data quality control according to claim 1, characterized in that: The use of the global initial model parameters to perform command-response consistency scoring on the local data samples of each client to evaluate the quality of each data sample is specifically as follows: f(q i ,a i ,the 0 )=L(a i ;θ 0 )-L((q i ,a i );i 0 ) Where f is the function for calculating data quality, i.e., high command-response consistency score; (q i , a i )∈D n , D n is the local client dataset of user n, n∈[1,N], N is the number of users participating in federated learning; L(a i θ 0 ) means that there is no instruction q i Generate response a in case i The loss, L((q i , a i );θ 0 ) indicates that in the given instruction q i Generate response a in case i The loss of θ 0 are the global initial model parameters.

3. The method for tuning federal instructions based on data quality control according to claim 1, characterized in that: The filtering of low-quality data according to the quality threshold, and only retaining data samples with a command-response consistency score higher than the quality threshold, is specifically as follows: D′ n ={(q v ,a i )∈D n |f(q i ,a i ,θ 0 )≥λ} Where D n is the local dataset of user n, D′ n is the filtered data set, and λ is the set quality threshold.

4. The method for tuning federal instructions based on data quality control according to claim 1, characterized in that: The filtered sample data is sorted from high to low according to the command-response consistency score and divided into multiple levels, each level containing the same number of data samples, specifically: Among them (q i , a i ) is taken from the data set after sorting in ascending order m is the filtered local data set D′ n The amount of data, k represents the level, and n represents the client.

5. The method for tuning federal instructions based on data quality control according to claim 1, characterized in that: Each client uses data samples in order of levels for local training, and gradually introduces data samples of lower-scoring levels, specifically: Client n selects the data H corresponding to the corresponding level k according to the training round nk Perform local training and update local model parameters Assume that the total number of training rounds is R, except for the last layer, the number of training rounds for each layer is The number of training rounds for the last layer is 6. The method for tuning federal instructions based on data quality control according to claim 5, characterized in that: Updating local model parameters is achieved through gradient descent: Where η is the learning rate, is the local loss function, For parameters The gradient of Initialize to t is the current number of local update steps, and the local update stops at t = T, and T is the number of steps in each round of local update.

7. The method for tuning federal instructions based on data quality control according to claim 1, characterized in that: After each round of training, the server receives the local updated parameters of all participating clients and updates the global model parameters by weighted average, specifically: Where |D n | is the data set D n The amount of data, The weights are usually assigned according to the size of each client’s dataset, S r is the set of users participating in this round of training. If all users participate in this round of training, then S r ={1, 2, ..., N}.

8. A federal instruction tuning system based on data quality control, characterized in that: include: Initialization module: The server sends global initial model parameters and quality thresholds to the client; Data quality assessment module: uses the global initial model parameters to perform command-response consistency scoring on the local data samples of each client to assess the quality of each data sample; Data filtering module: filtering low-quality data according to the quality threshold, and only retaining data samples whose command-response consistency score is higher than the quality threshold; Data sorting and stratification module: sorts the filtered sample data from high to low according to the command-response consistency score and divides it into multiple levels, each of which contains the same number of data samples; Hierarchical training module: Each client uses data samples in order of hierarchy for local training, and gradually introduces data samples of lower-scoring hierarchies; Global model aggregation module: After each round of training, the server receives the local updated parameters of all participating clients and updates the global model parameters by weighted averaging; Repeating module: Repeat the data quality assessment module, data filtering module, data sorting and stratification module, stratification training module and global model aggregation module until the preset training rounds are reached.

9. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it can be used to execute the method described in any one of claims 1 to 7, or to run the system described in claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it can be used to execute the method described in any one of claims 1 to 7, or to execute the system described in claim 8.

Citation Information

Patent Citations

  • Federation learning method, device and system based on multiple labels

    CN111160572A

  • Federal learning value evaluation-based chronic disease medical care method and system

    CN116598012A

  • Big data transaction and quality evaluation method and system taking big language model as medium

    CN117332247A

  • Model training method and device, data processing method and device, client and storage medium

    WO2022116502A1