Model training method and device, electronic equipment and storage medium

Abstract

The invention provides a model training method and device, electronic equipment and a storage medium, and the method comprises the steps: determining a to-be-trained global model, and determining a reference model obtained based on annotation data training; reasoning the annotation data based on the global model and the reference model to obtain a global reasoning result and a reference reasoning result of the annotation data, and determining an annotation deviation value of the global model based on the global reasoning result and the reference reasoning result of the annotation data and the tag of the annotation data; reasoning the unlabeled data based on the global model and the reference model to obtain a global reasoning result and a reference reasoning result of the unlabeled data, and performing parameter iteration on the global model based on the global reasoning result and the reference reasoning result of the unlabeled data and the labeling deviation value to complete model training. According to the model training method and device, the electronic equipment and the storage medium provided by the invention, the reference and global models are used for reasoning the unlabeled data, so that the use efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a model training method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of artificial intelligence technology, machine learning models based on large-scale data have been widely used in many fields, such as healthcare, financial services, and industrial Internet. However, there are significant challenges in obtaining labeled data in many scenarios: the labeling process requires a lot of manpower and time, and some fields (such as medical diagnosis and legal analysis) require extremely high labeling accuracy, which further increases the difficulty of data labeling. This makes unlabeled data a potential important resource.

[0003] In traditional machine learning, the utilization efficiency of unlabeled data is low, mainly because it lacks labels and cannot participate in training, resulting in a waste of data resources; models that rely on a small amount of labeled data have difficulty capturing the diversity of data distribution and have weak generalization capabilities; manual labeling costs are high, and the automatic pseudo-label generation mechanism is insufficient. In addition, the complexity and imbalance of the distribution of unlabeled data further increase the difficulty of utilization. Therefore, improving the utilization efficiency of unlabeled data is crucial to improving model performance. Summary of the invention

[0004] The present invention provides a model training method, device, electronic device and storage medium, which are used to solve the defect of low utilization efficiency of unlabeled data in related technologies.

[0005] The present invention provides a model training method, comprising the following steps: Determine a global model to be trained, and determine a reference model trained based on the labeled data; Reasoning the labeled data based on the global model and the reference model respectively to obtain a global reasoning result and a reference reasoning result of the labeled data, and determining a labeling deviation of the global model based on the global reasoning result and the reference reasoning result of the labeled data and the label of the labeled data; Reasoning is performed on the unlabeled data based on the global model and the reference model respectively to obtain global reasoning results and reference reasoning results of the unlabeled data, and based on the global reasoning results and reference reasoning results of the unlabeled data and the labeled deviation, parameter iteration is performed on the global model to complete model training.

[0006] According to a model training method provided by the present invention, the global reasoning result and reference reasoning result of the labeled data, as well as the label of the labeled data, are used to determine the label deviation of the global model, including: Based on a first difference between a global inference result of the labeled data and a label of the labeled data, and a second difference between a reference inference result of the labeled data and the label of the labeled data, a label deviation amount of the global model is determined.

[0007] According to a model training method provided by the present invention, the global inference result and the reference inference result of the unlabeled data, and the labeled deviation, performing parameter iteration on the global model, comprises: Determining a training loss value of the global model based on a difference between a global reasoning result of the unlabeled data and a reference reasoning result, and the labeling deviation; Based on the training loss value, parameters of the global model are iterated.

[0008] According to a model training method provided by the present invention, the step of determining a reference model obtained by training based on labeled data includes: Group the labeled data of each party in federated learning to obtain multiple groups of labeled data; Model training is performed based on multiple sets of labeled data to obtain multiple reference models.

[0009] According to a model training method provided by the present invention, when the federated learning is vertical federated learning, the labeled data of each party in the federated learning is grouped to obtain multiple groups of labeled data, including: Based on the labels of the labeled data at the initiator of the vertical federated learning, grouping the labeled data at the initiator to obtain multiple groups of labeled data at the initiator and sample identifiers in each group of labeled data; The sample identifiers in each group of labeled data are sent to each participant of the vertical federated learning, so that each participant groups the labeled data at each participant based on the sample identifiers in each group of labeled data.

[0010] According to a model training method provided by the present invention, when the federated learning is horizontal federated learning, the labeled data of each party in the federated learning is grouped to obtain multiple groups of labeled data, including: Receive the data volume and / or label number of the labeled data sent by each participant of the horizontal federated learning; Based on the data amount and / or label number of the labeled data at each participant, the labeled data at each participant is grouped to obtain multiple groups of labeled data.

[0011] According to a model training method provided by the present invention, based on the amount of the labeled data at each participant, the labeled data at each participant is grouped, including: Determine a data volume target value based on the number of groups and the data volume of the labeled data at each participant, and determine a current demand volume of each group of labeled data based on the data volume target value and the allocated data volume of each group of labeled data; Traverse the amount of ungrouped data of the labeled data at each participant, and when traversing to the current participant, traverse each group of labeled data, and when the amount of ungrouped data at the current participant is less than or equal to the current demand of the traversed current group of labeled data, allocate all the unallocated labeled data at the current participant to the current group of labeled data; Traverse the ungrouped data amount of the labeled data at each participant again, and when traversing to the current participant, traverse each group of labeled data, and when the ungrouped data amount at the current participant is less than or equal to the current demand amount of the traversed current group of labeled data, allocate all the unallocated labeled data at the current participant to the current group of labeled data; otherwise, allocate part of the unallocated labeled data at the current participant to the current group of labeled data, so that the allocated data amount of the current labeled data reaches the data amount target value.

[0012] According to a model training method provided by the present invention, the model training is performed based on multiple sets of labeled data to obtain multiple reference models, including: Cross-combining the multiple groups of labeled data to obtain multiple training sample sets, each of the training sample sets containing groups of labeled data with numbers different from the numbers of the training sample sets; Model training is performed based on each of the training sample sets to obtain a reference model corresponding to each training sample set.

[0013] The present invention also provides a model training device, comprising the following modules: A determination unit, used to determine a global model to be trained and a reference model trained based on the labeled data; A labeling application unit, configured to perform reasoning on the labeled data based on the global model and the reference model, respectively, to obtain a global reasoning result and a reference reasoning result of the labeled data, and to determine a labeling deviation of the global model based on the global reasoning result and the reference reasoning result of the labeled data and a label of the labeled data; The unlabeled application unit is used to reason about the unlabeled data based on the global model and the reference model, respectively, to obtain the global reasoning result and the reference reasoning result of the unlabeled data, and to iterate the parameters of the global model based on the global reasoning result and the reference reasoning result of the unlabeled data and the labeled deviation, so as to complete the model training.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the model training method described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the model training methods described above.

[0016] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the model training methods described above.

[0017] The model training method, device, electronic device and storage medium provided by the present invention respectively infer the unlabeled data through the reference model and the global model, generate the reference inference result and the global inference result of the unlabeled data, and combine the two to optimize the global model parameters, so that the unlabeled data can play a greater role in the training process. At the same time, it overcomes the limitation of the traditional model that only relies on labeled data training, effectively expands the scope of training data, and fully taps the potential of unlabeled data. At the same time, through the joint optimization of labeled data and unlabeled data, the generalization ability and prediction accuracy of the global model are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is one of the flow charts of the model training method provided by the present invention.

[0020] Figure 2 It is a schematic diagram of data grouping in the vertical federated learning scenario provided by the present invention.

[0021] Figure 3 It is a schematic diagram of data grouping in the horizontal federated learning scenario provided by the present invention.

[0022] Figure 4 It is a schematic diagram of the cross-training process provided by the present invention.

[0023] Figure 5 This is the second flow chart of the model training method provided by the present invention.

[0024] Figure 6 It is a structural schematic diagram of the model training device provided by the present invention.

[0025] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] With the rapid development of artificial intelligence technology, machine learning models based on large-scale data have been widely used in many fields such as healthcare, financial services, and industrial Internet. However, in practice, the acquisition of high-quality labeled data often faces huge challenges: the labeling process requires a lot of manpower, time, and resources, and some fields (such as medical imaging diagnosis, legal analysis, etc.) have extremely high requirements for the accuracy and professionalism of data labeling, which further increases the difficulty of data labeling. This makes the cost of labeling data high and the amount of data limited. At the same time, the utilization efficiency of unlabeled data is generally low, which has become an important bottleneck restricting the further improvement of machine learning models. In traditional machine learning methods, the utilization efficiency of unlabeled data is low, mainly because a large amount of unlabeled data cannot participate in model training due to lack of labels, and data resources are not fully utilized; models that rely only on a small amount of labeled data for training are difficult to fully capture the diversity of data distribution, resulting in weak model generalization ability; the cost of manual labeling of unlabeled data is too high, and there is a lack of effective automated mechanisms to generate high-quality pseudo-labels. In addition, the distribution of unlabeled data itself may be complex or uneven, which increases the difficulty of models using these data. Therefore, improving the utilization efficiency of unlabeled data has become the key to improving model performance.

[0028] In order to solve the above problems, an embodiment of the present invention provides a model training method. Figure 1 This is one of the flow charts of the model training method provided by the present invention. Figure 1 As shown, the method includes: Step 110, determining a global model to be trained, and determining a reference model trained based on the labeled data.

[0029] Specifically, the global model is the model to be trained. For example, in a federated learning scenario, the global model can be a model that is initialized with parameters and deployed at each participant. In a federated learning scenario, the training of the global model is based on the aggregation of model parameters provided by different participants, and its final output is used for unified model application and reasoning.

[0030] The reference model is a model trained based on labeled data. It is optimized directly on the labeled data and provides reliable performance evaluation and feature description. The reference model can be used to help train subsequent global models and provide a basis for generating pseudo labels for reasoning about unlabeled data. During the training process, the performance results of the reference model can help analyze and correct the parameters of the global model to further improve the performance of the global model.

[0031] Here, the global model is the model to be trained, and the reference model is the model that has been trained. The global model and the reference model are designed to achieve the same function. For example, in the field of financial services, the global model and the reference model can both be a profiling model based on personnel credit data, or a credit risk assessment model based on personnel credit data.

[0032] Step 120, inferring the labeled data based on the global model and the reference model respectively to obtain the global reasoning result and the reference reasoning result of the labeled data, and determining the labeling deviation of the global model based on the global reasoning result and the reference reasoning result of the labeled data and the label of the labeled data.

[0033] First, the global model is used to infer the labeled data to generate the global inference results of the labeled data. The inference results reflect the prediction ability of the global model on the labeled data and provide a basis for the subsequent calculation of the deviation. At the same time, the labeled data is inferred based on the reference model to generate the reference inference results of the labeled data. The reference model is trained based on the labeled data, so its inference results are considered to have high reliability and can be used as a benchmark for evaluating the performance of the global model.

[0034] Next, the global inference results and the reference inference results are compared with the labels of the annotated data, and the error values ​​are calculated using a preset loss function (such as mean square error or cross entropy). The error between the global inference result and the true label reflects the direct performance of the global model on the annotated data, while the error between the reference inference result and the true label measures the prediction ability of the reference model for the annotated data. By analyzing the difference between the global inference result and the reference inference result, and combining the error values ​​of the two with the true label, the annotation deviation of the global model on the annotated data can be calculated. The annotation deviation is a key indicator to measure the performance of the global model on the annotated data, which is used to reflect the accuracy gap between the global model and the reference model. The determination of this deviation provides a basis for the subsequent correction of the global model parameters, thereby helping the global model to gradually optimize its performance on the annotated data and improve its adaptability to the distribution of real data. Through this process, the possible deficiencies of the global model on the annotated data can be effectively identified, and guidance can be provided for improving the model performance.

[0035] Step 130, respectively reasoning the unlabeled data based on the global model and the reference model to obtain the global reasoning result and the reference reasoning result of the unlabeled data, and based on the global reasoning result and the reference reasoning result of the unlabeled data and the labeled deviation, performing parameter iteration on the global model to complete model training.

[0036] Specifically, by reasoning on the unlabeled data through the global model, a global reasoning result of the unlabeled data can be generated. This result is based on the current parameters of the global model and reflects the preliminary prediction ability of the global model on the unlabeled data. At the same time, the reference model is used to reason on the unlabeled data to generate a reference reasoning result of the unlabeled data. The reference reasoning result can essentially be regarded as a pseudo-label of the unlabeled data. They are generated by the reference model trained based on the labeled data and provide label estimates for the unlabeled data.

[0037] Next, the deviation of the unlabeled data can be calculated by comparing the global inference results based on the unlabeled data with the reference inference results (i.e., pseudo labels). The deviation of the unlabeled data reflects the performance of the global model on the unlabeled data and can provide feedback information on the optimization of the global model by the unlabeled data. At the same time, combined with the labeled deviation calculated in the previous step, it represents the error performance of the global model on the labeled data. The comprehensive deviation of the global model is generated by adding the deviation of the unlabeled data to the labeled deviation. This comprehensive deviation indicates the overall performance of the global model on the labeled and unlabeled data and serves as the basis for parameter iteration. The parameters of the global model can be updated with the goal of minimizing the comprehensive deviation. With the progress of multiple iterations, the parameters of the global model are gradually optimized, and finally achieve good adaptability on the joint distribution of labeled and unlabeled data.

[0038] The model training method provided by the present invention uses a reference model and a global model to infer unlabeled data respectively, generates reference inference results and global inference results of unlabeled data, and combines the two to optimize the global model parameters, so that unlabeled data can play a greater role in the training process. At the same time, it overcomes the limitation of traditional models that only rely on labeled data for training, effectively expands the scope of training data, and fully taps the potential of unlabeled data. At the same time, through the joint optimization of labeled data and unlabeled data, the generalization ability and prediction accuracy of the global model are improved.

[0039] Based on the above embodiment, in step 120, determining the annotation deviation of the global model based on the global reasoning result and the reference reasoning result of the annotated data, and the label of the annotated data includes: Based on a first difference between a global inference result of the labeled data and a label of the labeled data, and a second difference between a reference inference result of the labeled data and the label of the labeled data, a label deviation amount of the global model is determined.

[0040] Specifically, the difference between the calculated global inference result and the true label of the labeled data is recorded as the first difference. The first difference is used to quantify the prediction error of the global model on the labeled data, which is a direct reflection of the global model's ability to predict the labeled data. The calculation formula for the first difference is: in, For the first difference, The loss function used by the global model is used to measure the deviation between the inference result of the global model and the true label. and Represent the dataset of labeled data and the true label of labeled data respectively, are the parameters of the global model.

[0041] At the same time, the difference between the reference inference result obtained by inferring the labeled data using the reference model and the true label of the labeled data is recorded as the second difference. The second difference is used to measure the prediction error of the reference model on the labeled data, which can help evaluate the performance reliability of the global model. The calculation formula of the second difference is: in, For the second difference, is the reference reasoning result, where The ordinal number of the reference model, The value range is 1 to , is the number of reference models. The number of reference models is determined by the sample size of the labeled data or the number of groups of the sample data.

[0042] Combined with the first difference and the second difference , the annotation deviation of the global model can be calculated: The annotation deviation is calculated by subtracting the error of the global model from the error of the reference model to quantify the deviation of the global model's reasoning performance from that of the reference model. The specific calculation formula for the annotation deviation is: in, is the total amount of labeled data, and are the i-th labeled data and its label respectively, It is The inference result of the reference model based on the i-th labeled data. By comparing the first difference and the second difference of each sample based on each reference model one by one, the average deviation of all labeled data samples is calculated, reflecting the reasoning ability of the global model on the labeled data and its gap with the reasoning ability of the reference model on the labeled data.

[0043] Based on the above embodiment, in step 130, the global inference result and the reference inference result of the unlabeled data, as well as the labeled deviation, are used to iterate the parameters of the global model, including: Determining a training loss value of the global model based on a difference between a global reasoning result of the unlabeled data and a reference reasoning result, and the labeling deviation; Based on the training loss value, parameters of the global model are iterated.

[0044] Specifically, the current global model first generates global inference results for unlabeled data based on the unlabeled data, reflecting the reasoning ability of the global model on the unlabeled data. The reference inference results for the unlabeled data are generated by the reference model and serve as pseudo labels for the unlabeled data, providing relatively reliable label estimates for the unlabeled data. Next, the difference between the global inference results for the unlabeled data and the reference inference results is calculated, i.e., the deviation of the unlabeled data, to measure the degree of deviation between the global model inference results and the reference model benchmark. This difference is usually calculated using a loss function: in, is the deviation of the unlabeled data, is the i-th unlabeled data, is the reference model inference result of the i-th unlabeled data, that is, the pseudo label of the i-th unlabeled data.

[0045] Based on the deviation of the unlabeled data, the difference is combined with the labeled deviation to comprehensively determine the training loss value of the global model. The labeled deviation reflects the performance error of the global model on the labeled data, and its introduction helps to comprehensively evaluate the overall performance of the global model. Specifically, the training loss value of the global model can be obtained by adding the deviation of the unlabeled data and the labeled deviation: in, is the training loss value of the global model. You can also consider summing up the deviation of unlabeled data and the deviation of labeled data in a weighted manner to obtain the training loss value: in, and are the weight parameters for the bias of unlabeled data and the bias of labeled data, respectively, which are used to balance the contribution of unlabeled data and labeled data to training.

[0046] Based on the training loss value, you can use optimization algorithms such as gradient descent to iteratively update the global model parameters. For example, in each iteration, calculate the gradient of the training loss value relative to the global model parameters, and use this as a basis to adjust the direction and amplitude of the parameters so that the prediction error of the global model is gradually reduced. After multiple iterations, the performance of the global model on labeled and unlabeled data will be significantly improved, and the training process will eventually be completed.

[0047] Through the above process, the global model can effectively combine labeled data and unlabeled data for optimization, which not only fully utilizes the potential of unlabeled data, but also improves the overall accuracy and generalization ability of the model through the guidance of labeled deviation.

[0048] With the widespread application of machine learning technology, model training based on distributed data has become a mainstream method in many fields. However, in practical applications, the data distribution between participants usually has significant heterogeneity, and this difference in data distribution has a profound impact on the stability and performance of model training. Traditional federated learning or distributed training methods usually ignore the heterogeneity of data distribution and adopt a unified model training strategy. This training method cannot effectively deal with the data differences between participants and may lead to a decline in global model performance. In addition, data heterogeneity also increases the difficulty and communication cost of model training, especially when the amount of data or label distribution of the participants is very different, the optimization process of the model will have problems such as slow convergence or even instability.

[0049] To address the above problems, the present invention introduces a model training method based on data grouping. By grouping the labeled data, the data is divided into multiple groups according to categories or feature distributions, so as to better adapt to the distribution characteristics of the data.

[0050] Based on the above embodiment, in step 110, determining a reference model trained based on the labeled data includes: Group the labeled data of each party in federated learning to obtain multiple groups of labeled data; Model training is performed based on multiple sets of labeled data to obtain multiple reference models.

[0051] Specifically, federated learning is an emerging distributed machine learning framework that allows multiple parties to jointly train models without sharing data. This framework achieves efficient collaboration of distributed data by protecting data privacy, and is an important solution to the problem of data silos. In the federated learning scenario, the data of each participant usually have different distributions, and this heterogeneity poses challenges to model training. In particular, when the amount of labeled data is limited and unevenly distributed, how to efficiently use this data and build a reliable reference model becomes the key.

[0052] In the embodiment of the present invention, in order to use the labeled data more efficiently, the labeled data is grouped. The grouping process can be based on a variety of methods, such as grouping the labeled data with the same category according to the category label of the labeled data; using a clustering algorithm to aggregate the labeled data according to feature similarity, etc. The purpose of grouping the labeled data is to organize the data more evenly and structured, so that each group of labeled data can concentrate on reflecting specific distribution characteristics.

[0053] Subsequently, the reference models are trained based on the grouped labeled data. Each group of data is used to train an independent reference model, and these reference models can accurately capture the characteristics of each group of labeled data. The training process can adopt common machine learning methods or deep learning frameworks, and use standard optimization algorithms to gradually adjust the model parameters so that it can achieve a higher degree of fit on the labeled data of the current group. The training method of the reference model can be flexibly selected, such as the cross entropy loss commonly used in classification tasks or the mean square error in regression tasks, which is not limited by the present invention. The reference models reflect the characteristics of each group of labeled data respectively, and provide a reliable benchmark for subsequent reasoning and optimization. These reference models can not only achieve good performance within the local data range, but also provide important support for the training optimization of the global model, such as the reference model participates in calculating the labeled deviation or generating the reasoning results of unlabeled data.

[0054] Based on the above embodiment, when the federated learning is vertical federated learning, the labeled data of each party in the federated learning is grouped to obtain multiple groups of labeled data, including: Based on the labels of the labeled data at the initiator of the vertical federated learning, grouping the labeled data at the initiator to obtain multiple groups of labeled data at the initiator and sample identifiers in each group of labeled data; The sample identifiers in each group of labeled data are sent to each participant of the vertical federated learning, so that each participant groups the labeled data at each participant based on the sample identifiers in each group of labeled data.

[0055] Specifically, in the data collaboration scenario, vertical federated learning refers to a scenario where the data between the participants overlap in samples but have different feature dimensions, that is, different participants hold different features of the same sample. For example, one hospital may have basic demographic information and diagnostic data of a patient, while another hospital may hold genetic or imaging data of the same patient. In this case, the parties need to find the same sample set through sample alignment, and then combine different features together for joint modeling. The core of vertical federated learning is to fully explore the complementarity of multi-source data through data collaboration of different feature dimensions while protecting data privacy.

[0056] Figure 2 Schematic diagram of data grouping in the vertical federated learning scenario provided by the present invention. Figure 2 As shown in the figure, the initiator divides the data into multiple groups (such as the first group, the second group, and the third group) according to the labels of the labeled data, and assigns a sample identifier to each group of data. Each participant groups the corresponding sample data locally according to the sample identifier provided by the initiator, so as to achieve grouping consistency. At the same time, for unlabeled data, each participant does not participate in grouping.

[0057] Specifically, the initiator divides the data into multiple groups according to the category labels of the annotated data. When grouping, the initiator ensures that each group of data contains a certain proportion of samples of each category to achieve category diversity and balanced distribution of data within the group. For example, for a classification task, the initiator can randomly distribute the data to multiple groups according to the category ratio, so that each group contains both samples of the first category and samples of the second category and other categories. Through such a grouping method, the scientificity and rationality of data grouping can be guaranteed, while improving the representativeness of the grouped data. After the grouping is completed, the initiator directly sends the sample identification information of each group of data to each participant. The sample identification is a number used to uniquely identify each sample, which can clearly indicate the specific samples contained in each group. After receiving these sample identifications, the participants group their local data according to the grouping information of the initiator. Specifically, the participants extract the corresponding sample feature data from the local data according to the received sample identification, and organize them according to the received grouping information. For example, the data features belonging to the first group of sample identifications are stored in the first group, the data features belonging to the second group of sample identifications are stored in the second group, and so on.

[0058] For example, in a vertical federated learning scenario, each sample in the initiator is numbered j Unique identifier, the initiator’s j The labeled data of a sample is represented as ,in, j The value range is 1 to N , For the initiator jThe data characteristics of the samples, For the initiator j The labels of samples of other participants. j The samples are represented as The initiator's labeled data will first be grouped by category. Assuming the initiator's labeled data label The number of categories included is 3, and the label of each category is c , c The value range of is 1 to 3. The initiator first separates the labeled data by category. For example, the data set belonging to category 1 is recorded as Then, the labeled data of each category is randomly divided into K In each group, G k Contains a certain proportion of labeled data for this category. For example, each group G k It contains 30% of the data in category 1, 40% of the data in category 2, and 30% of the data in category 3. After the grouping is completed, the initiator will group the sample numbers of each group of data. G k Sent to each participant. Based on these numbers, the participant extracts the corresponding data features from its local data. . And divided into groups, so as to be consistent with the initiator's data grouping. Finally, the initiator and the participants' data are divided into K Each group of data is aligned among all participants and contains different dimensional features of the same samples.

[0059] By directly sending sample identifiers, the initiator and all participants achieve consistency in data grouping and sample alignment. Ultimately, each group of data remains fully aligned among all participants, that is, each group of data contains different feature dimensions of the same sample. This grouping and synchronization mechanism not only makes full use of the label information of the annotated data, but also achieves efficient collaboration between the participants, providing a stable foundation for subsequent joint model training, while improving the training effect and prediction performance of the model under diverse data distribution.

[0060] Based on the above embodiment, when the federated learning is horizontal federated learning, the labeled data of each party in the federated learning is grouped to obtain multiple groups of labeled data, including: Receive the data volume and / or label number of the labeled data sent by each participant of the horizontal federated learning; Based on the data amount and / or label number of the labeled data at each participant, the labeled data at each participant is grouped to obtain multiple groups of labeled data.

[0061] In the data collaboration scenario, horizontal federated learning refers to the scenario where the data between the participants are the same in feature dimensions, but the sample sets do not overlap, that is, different participants hold the same features of different samples. For example, two banks may hold their own customer transaction records and credit score data respectively, but the feature dimensions of their data are consistent, such as income, consumption amount, etc. In this case, each participant uses its own and other parties' data to jointly improve the performance of the global model through collaborative training without directly sharing the original data. The core of horizontal federated learning is to expand the sample coverage of training data and improve the generalization ability of the model through joint modeling of multi-party data, while protecting the data privacy of all parties from being leaked through a distributed training mechanism. This learning method has important application value in industries such as finance and insurance.

[0062] In horizontal federated learning, all participants need to collaborate to complete the grouping of labeled data in order to provide balanced and representative grouped data for subsequent collaborative training. In this process, each participant needs to send relevant information of its labeled data to a collaborative node, such as a central server, or send its labeled data to other participants. This information includes data volume and / or label number, and the content to be sent is selected according to actual needs: if only data volume information is sent, the grouping is mainly based on the balanced distribution of data scale; if only label number information is sent, the grouping will optimize the category balance based on the label distribution; if both data volume and label number information are provided, the grouping can achieve data scale balance and category diversity optimization at the same time. In this way, all participants collaborate to complete the grouping of labeled data without sharing the original data.

[0063] For example, in the case of hierarchical grouping, each participant synchronizes its sample quantity and label number to a central server or collaboration node. The central node sorts the global data by category based on the received label number information, and evenly distributes samples of different categories to multiple groups. The grouped results are returned to each participant, and the participant extracts local data based on the grouping information. This process is similar to hierarchical grouping in vertical federated learning, except that in horizontal federated learning, all participants hold labeled data. For example, Bank A and Bank B participate in horizontal federated learning, and Bank A's data is mainly "low risk" and "medium risk", while Bank B's data is mainly "medium risk" and "high risk". The central node receives the data volume and label number information of both parties, ensuring that the "low risk", "medium risk" and "high risk" data are evenly distributed to multiple groups, and each group of data covers samples of different categories. This method ensures the balance of data distribution, but requires global synchronization of information, and the communication overhead is large. It is suitable for scenarios with high requirements for data distribution uniformity.

[0064] For another example, in the local random grouping method, each participant completes the grouping independently, and there is no need to synchronize the number of samples or label number information. Each participant randomly divides the data by label category locally. This process is the same as the logic of the local hierarchical grouping of participants in vertical federated learning, except that in horizontal federation, the data grouping of each participant does not need to be aligned across parties. Since there is no global synchronization information involved, the grouping speed is fast and the communication overhead is low, but the global data may be uneven after grouping. For example, in an insurance company cooperation scenario, the data of company A is mainly "low premium", and the data of company B covers "medium premium" and "high premium". After the two companies complete the random grouping locally, there may be significant differences in the label categories of the groups. For example, some groups may contain a large number of "low premium" users, while there are fewer "high premium" users. This method is suitable for scenarios that are sensitive to communication resources, but the balance of the grouping results depends on the data distribution of the participants.

[0065] After sending the amount of labeled data, if the amount of labeled data of each participant is relatively balanced, the exclusion mechanism grouping method is adopted. Under the exclusion mechanism grouping method, the number of groups 𝐾 is equal to the number of participants 𝑀, and the data of each participant is directly divided into an independent group. For example, Hospital A, Hospital B, and Hospital C participate in horizontal federated learning, and the data of the three hospitals are directly used as Group 1, Group 2, and Group 3 respectively. This method does not involve sample information synchronization or complex grouping logic, and has the lowest communication cost, but the data distribution depends entirely on the original characteristics of each participant. For example, if the data of a hospital is concentrated in a specific category, it may cause the data in this group to be single, which is not conducive to the generalization ability of the global model. This method is suitable for scenarios where the amount of data and the distribution of categories are balanced. At the same time, due to the simple grouping, the cost of subsequent training is low.

[0066] Figure 3 Schematic diagram of data grouping in the horizontal federated learning scenario provided by the present invention. Figure 3 As shown in the figure, the sample data of participant 0 is divided into multiple groups, such as the first and third groups, and there are also unlabeled data, which do not participate in the grouping. Participant 1 only contains unlabeled data, so it is not assigned to any group. The sample data of participant 2 is all assigned to one group (the second group), and all data are concentrated in this group. The sample data of participant 3 is divided into multiple groups, such as the third and fourth groups. The data are all labeled and can fully participate in the group training. Through data grouping, the problem of uneven data distribution between participants can be effectively alleviated in horizontal federated learning. Grouping can divide the data into more balanced subsets, ensuring that each group of data is more reasonable in terms of category coverage and sample quantity, thereby reducing the deviation of model training. Grouping can also improve training efficiency. After dividing the global data into multiple smaller groups, training or optimization can be performed independently on each group, while supporting parallel processing of each group to shorten training time.

[0067] Based on the above embodiment, grouping the labeled data at each participant based on the amount of the labeled data at each participant includes: Determine a data volume target value based on the number of groups and the data volume of the labeled data at each participant, and determine a current demand volume of each group of labeled data based on the data volume target value and the allocated data volume of each group of labeled data; Traverse the amount of ungrouped data of the labeled data at each participant, and when traversing to the current participant, traverse each group of labeled data, and when the amount of ungrouped data at the current participant is less than or equal to the current demand of the traversed current group of labeled data, allocate all the unallocated labeled data at the current participant to the current group of labeled data; Traverse the ungrouped data amount of the labeled data at each participant again, and when traversing to the current participant, traverse each group of labeled data, and when the ungrouped data amount at the current participant is less than or equal to the current demand amount of the traversed current group of labeled data, allocate all the unallocated labeled data at the current participant to the current group of labeled data; otherwise, allocate part of the unallocated labeled data at the current participant to the current group of labeled data, so that the allocated data amount of the current labeled data reaches the data amount target value.

[0068] Specifically, according to the set number of groups and the amount of labeled data of each participant, the target data volume value that each group of data needs to reach is determined. This target value is the result of a balanced distribution of the total amount of global labeled data, ensuring that the amount of data for each group is as close to the target value as possible. Then, based on the gap between the currently allocated amount of data for each group of labeled data and the target amount of data, the current demand for each group of labeled data is calculated to guide the next allocation operation.

[0069] After determining the target data volume and the demand for each group of labeled data, the preliminary allocation phase begins. When traversing to a certain participant, an allocation attempt is made to the participant's data, and the data is preferentially allocated to the group with the current larger demand. If the amount of ungrouped data of the participant is less than or equal to the demand of the current group, all the data of the participant is directly allocated to the group, increasing the amount of allocated data for the group; if the amount of ungrouped data is greater than the demand of the current group, the remaining data is retained for further allocation.

[0070] After the initial allocation, some data may still not be allocated to the group, so the second stage of supplementary allocation is entered. In this stage, the amount of ungrouped data of each participant is traversed again, and attempts are continued to allocate data to unfilled groups. When traversing to a certain group, if the amount of ungrouped data of the current participant is less than or equal to the current demand of the group, all of it is allocated to the group so that the data volume of the group is as close to the target value as possible. If the amount of ungrouped data is greater than the demand of the current group, only part of the data is allocated to the group so that the data volume of the group just reaches the target value, and the remaining data is retained to continue to try to allocate to other groups. The allocation strategy at this stage can maximize the data volume requirements of each group while ensuring the overall balance of the grouping process.

[0071] This grouping method can be implemented by the minimum delivery number greedy algorithm. Assume that the number of data groups is K Group, the number of participants is M , the amount of labeled data of each participant is ls. The goal is to reasonably distribute the labeled data of each participant to K In each group, the total amount of data in each group is as balanced as possible. At the beginning of the algorithm, the target number of samples needed per group needs_per_group is calculated, and its value is the integer division result of the total amount of data of all participants sum(ls) / / K . Next, sort the data of the participants from large to small according to the number of samples to form a sorted list ls_sorted, ensuring that the data of participants with large sample sizes are allocated first. Initialize a two-dimensional array result to record the number of allocations of each participant in each group. The array size is K×M, and the initial value is zero. At the same time, initialize a list received with a length of K to track the total amount of labeled data currently obtained by each group. The initial value is also zero. The first stage of allocation begins, traversing the sorted participant sample data ls_sorted. For each labeled data amount stock and its corresponding index 𝑗, try to allocate it to each group of data. For each group i ( i For each group index), calculate the current remaining demand of the group: remaining_need = need_per_group-group_received( i ). If the stock is less than or equal to the remaining demand of the current group and greater than zero, all the stock is allocated to the group. The specific operation includes subtracting the stock from remaining_need and adding the stock to group_received ( i ), and record the allocation result[ i ][ j]=stock. If the stock is greater than the remaining demand of the current group, skip this group and try to allocate it to the next group. After the first stage of allocation, some data of some participants may still not be fully allocated, so enter the second stage of supplementary allocation. In this stage, ls_sorted is re-traversed to allocate the remaining data. For each participant's stock and each group i , calculate the remaining demand remaining_need again. If stock is less than or equal to remaining_need, directly allocate stock to the group; otherwise, allocate part of the data to the group so that the amount of data in the group just reaches the target value need_per_group. At this time, update group_received( i ) and result data result[ i ][ j ], while reducing the remaining amount of stock.

[0072] Through the traversal and allocation in the above two stages, the algorithm realizes the reasonable grouping of the labeled data of each participant. In the end, the allocation amount of each group of labeled data will be as close to the target data amount as possible, and the labeled data of all participants will be allocated to the appropriate group.

[0073] Based on the above embodiment, the model training is performed based on multiple sets of labeled data to obtain multiple reference models, including: Cross-combining the multiple groups of labeled data to obtain multiple training sample sets, each of the training sample sets containing groups of labeled data with numbers different from the numbers of the training sample sets; Model training is performed based on each of the training sample sets to obtain a reference model corresponding to each training sample set.

[0074] Specifically, each training sample set is numbered i , excluding i The design aims to reduce the model's dependence on a single group of data and improve the model's generalization ability by alternating between different grouped data. Figure 4 Schematic diagram of the cross-training process provided by the present invention. Figure 4As shown, the third group of data of participant 0, the second group of data of participant 2, and the third and fourth groups of data of participant 3 constitute the first training sample combination; the first group of data and the third group of data of participant 0, and the third and fourth groups of data of participant 3 constitute the second training sample combination; the first group of data of participant 0, the second group of data of participant 2, and the fourth group of data of participant 3 constitute the third training sample combination; the first group and the third group of data of participant 0, the second group of data of participant 2, and the third group of data of participant 3 constitute the fourth training sample combination.

[0075] Next, independent model training is performed for each training sample set. The training of each model is based only on the data contained in the current training sample set, and does not involve the data of the group with the same set number. For example, in Figure 4 In the first training, only the first training sample combination is used for training, without using the first set of data from participant 0. The model trained in this way can more comprehensively capture the characteristics of different data combinations, while effectively avoiding overfitting of the model to a specific data set and improving the overall generalization ability.

[0077] This cross-combination and independent training mechanism ensures that each set of data is fully utilized in different combinations, thereby improving the generalization ability and diversity of the model. Multiple reference models reflect the characteristics of different training sample sets. These models provide diverse information support in the subsequent global model training, further enhancing the comprehensive performance and robustness of the global model. This method not only improves data utilization by flexibly combining and alternating grouped data, but also significantly optimizes the training effect of the model, laying an important foundation for subsequent model integration and optimization.

[0078] Based on the above embodiments, Figure 5 This is the second flow chart of the model training method provided by the present invention. Figure 5 The complete process of the federated learning model from data preparation to model training and reasoning is shown. Figure 5 As shown in Figure 1, first, the original training set is received. These data may come from multiple parties, and due to privacy protection requirements, the data is not directly shared.

[0079] Next, enter the federated learning model training module. First, group the labeled data. This step divides the labeled data into multiple groups. The purpose is to organize the data according to specific rules (such as label categories or data characteristics) to prepare for subsequent distributed collaborative training. After the grouping is completed, enter the federated cross-prediction model training phase. In this stage, each participant uses the grouped labeled data for collaborative training. The core of cross-training is to ensure that each group of data can participate in training by alternating different grouped data. For example, in one round of training, part of the grouped data is used to optimize the model, and in the next round, it switches to another part of the grouped data, thereby reducing the model's dependence on a single data group and enabling it to learn more diverse data characteristics. Subsequently, the participants integrate the training results through the federated learning mechanism to further optimize the global model. Subsequently, enter the federated learning global model generation phase, which contains multiple sub-steps. First, data label inference is performed through the results of federated cross-prediction model training to generate pseudo labels for unlabeled data. Then, the federated deviation is calculated and the prediction error of the global model is corrected by combining the local model results of each participant. Finally, the training results of all participants are integrated to generate the final global federated model, which represents the comprehensive characteristics of data from multiple participants and has stronger generalization capabilities.

[0080] Finally, the generated global federated model is used for reasoning to complete specific prediction tasks.

[0081] The model training device provided by the present invention is described below. The model training device described below and the model training method described above can be referenced to each other.

[0082] Figure 6 It is a structural schematic diagram of the model training device provided by the present invention, such as Figure 3 As shown, the device comprises: A determination unit 610, configured to determine a global model to be trained, and to determine a reference model trained based on the labeled data; The annotation application unit 620 is used to perform reasoning on the annotated data based on the global model and the reference model respectively to obtain a global reasoning result and a reference reasoning result of the annotated data, and determine the annotation deviation of the global model based on the global reasoning result and the reference reasoning result of the annotated data and the label of the annotated data; The unlabeled application unit 630 is used to infer the unlabeled data based on the global model and the reference model, respectively, to obtain the global reasoning result and the reference reasoning result of the unlabeled data, and to iterate the parameters of the global model based on the global reasoning result and the reference reasoning result of the unlabeled data and the labeled deviation to complete the model training.

[0083] The model training device provided by the present invention uses a reference model and a global model to infer unlabeled data respectively, generates reference inference results and global inference results of unlabeled data, and combines the two to optimize global model parameters, so that unlabeled data can play a greater role in the training process. At the same time, it overcomes the limitation of traditional models that only rely on labeled data for training, effectively expands the scope of training data, and fully taps the potential of unlabeled data. At the same time, through the joint optimization of labeled data and unlabeled data, the generalization ability and prediction accuracy of the global model are improved.

[0084] Based on any of the foregoing embodiments, the determining unit is specifically configured to: Group the labeled data of each party in federated learning to obtain multiple groups of labeled data; Model training is performed based on multiple sets of labeled data to obtain multiple reference models.

[0085] Based on the labels of the labeled data at the initiator of the vertical federated learning, grouping the labeled data at the initiator to obtain multiple groups of labeled data at the initiator and sample identifiers in each group of labeled data; The sample identifiers in each group of labeled data are sent to each participant of the vertical federated learning, so that each participant groups the labeled data at each participant based on the sample identifiers in each group of labeled data.

[0086] Receive the data volume and / or label number of the labeled data sent by each participant of the horizontal federated learning; Based on the data amount and / or label number of the labeled data at each participant, the labeled data at each participant is grouped to obtain multiple groups of labeled data.

[0087] Determine a data volume target value based on the number of groups and the data volume of the labeled data at each participant, and determine a current demand volume of each group of labeled data based on the data volume target value and the allocated data volume of each group of labeled data; Traverse the amount of ungrouped data of the labeled data at each participant, and when traversing to the current participant, traverse each group of labeled data, and when the amount of ungrouped data at the current participant is less than or equal to the current demand of the traversed current group of labeled data, allocate all the unallocated labeled data at the current participant to the current group of labeled data; Traverse the ungrouped data amount of the labeled data at each participant again, and when traversing to the current participant, traverse each group of labeled data, and when the ungrouped data amount at the current participant is less than or equal to the current demand amount of the traversed current group of labeled data, allocate all the unallocated labeled data at the current participant to the current group of labeled data; otherwise, allocate part of the unallocated labeled data at the current participant to the current group of labeled data, so that the allocated data amount of the current labeled data reaches the data amount target value.

[0088] Cross-combining the multiple groups of labeled data to obtain multiple training sample sets, each of the training sample sets containing groups of labeled data with numbers different from the numbers of the training sample sets; Model training is performed based on each of the training sample sets to obtain a reference model corresponding to each training sample set.

[0089] Based on any of the above embodiments, the annotation application unit is specifically used for: Based on a first difference between a global inference result of the labeled data and a label of the labeled data, and a second difference between a reference inference result of the labeled data and the label of the labeled data, a label deviation amount of the global model is determined.

[0090] Based on any of the foregoing embodiments, the unlabeled application unit is specifically used for: Determining a training loss value of the global model based on a difference between a global reasoning result of the unlabeled data and a reference reasoning result, and the labeling deviation; Based on the training loss value, parameters of the global model are iterated.

[0091] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the model training method, which includes: Determine a global model to be trained, and determine a reference model trained based on the labeled data; Reasoning the labeled data based on the global model and the reference model respectively to obtain a global reasoning result and a reference reasoning result of the labeled data, and determining a labeling deviation of the global model based on the global reasoning result and the reference reasoning result of the labeled data and the label of the labeled data; Reasoning is performed on the unlabeled data based on the global model and the reference model respectively to obtain global reasoning results and reference reasoning results of the unlabeled data, and based on the global reasoning results and reference reasoning results of the unlabeled data and the labeled deviation, parameter iteration is performed on the global model to complete model training.

[0092] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0093] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the model training method provided by the above methods, the method includes: Determine a global model to be trained, and determine a reference model trained based on the labeled data; Reasoning the labeled data based on the global model and the reference model respectively to obtain a global reasoning result and a reference reasoning result of the labeled data, and determining a labeling deviation of the global model based on the global reasoning result and the reference reasoning result of the labeled data and the label of the labeled data; Reasoning is performed on the unlabeled data based on the global model and the reference model respectively to obtain global reasoning results and reference reasoning results of the unlabeled data, and based on the global reasoning results and reference reasoning results of the unlabeled data and the labeled deviation, parameter iteration is performed on the global model to complete model training.

[0094] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the model training method provided by the above methods, the method comprising: Determine a global model to be trained, and determine a reference model trained based on the labeled data; Reasoning the labeled data based on the global model and the reference model respectively to obtain a global reasoning result and a reference reasoning result of the labeled data, and determining a labeling deviation of the global model based on the global reasoning result and the reference reasoning result of the labeled data and the label of the labeled data; Reasoning is performed on the unlabeled data based on the global model and the reference model respectively to obtain global reasoning results and reference reasoning results of the unlabeled data, and based on the global reasoning results and reference reasoning results of the unlabeled data and the labeled deviation, parameter iteration is performed on the global model to complete model training.

[0095] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0096] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A model training method, characterized in that: include: Determine a global model to be trained, and determine a reference model trained based on the labeled data; Reasoning the labeled data based on the global model and the reference model respectively to obtain a global reasoning result and a reference reasoning result of the labeled data, and determining a labeling deviation of the global model based on the global reasoning result and the reference reasoning result of the labeled data and the label of the labeled data; Reasoning is performed on the unlabeled data based on the global model and the reference model respectively to obtain global reasoning results and reference reasoning results of the unlabeled data, and based on the global reasoning results and reference reasoning results of the unlabeled data and the labeled deviation, parameter iteration is performed on the global model to complete model training.

2. The model training method according to claim 1, characterized in that: The step of determining the annotation deviation of the global model based on the global reasoning result and the reference reasoning result of the annotated data and the label of the annotated data comprises: Based on a first difference between a global inference result of the labeled data and a label of the labeled data, and a second difference between a reference inference result of the labeled data and the label of the labeled data, a label deviation amount of the global model is determined.

3. The model training method according to claim 1, characterized in that: The performing parameter iteration on the global model based on the global reasoning result and the reference reasoning result of the unlabeled data and the labeled deviation includes: Determining a training loss value of the global model based on a difference between a global reasoning result of the unlabeled data and a reference reasoning result, and the labeling deviation; Based on the training loss value, parameters of the global model are iterated.

4. The model training method according to any one of claims 1 to 3, characterized in that: The determining of a reference model obtained by training based on the labeled data includes: Group the labeled data of each party in federated learning to obtain multiple groups of labeled data; Model training is performed based on multiple sets of labeled data to obtain multiple reference models.

5. The model training method according to claim 4, characterized in that: In the case where the federated learning is vertical federated learning, the labeled data of each party in the federated learning is grouped to obtain multiple groups of labeled data, including: Based on the labels of the labeled data at the initiator of the vertical federated learning, the labeled data at the initiator is grouped to obtain multiple groups of labeled data at the initiator and sample identifiers in each group of labeled data; The sample identifiers in each group of labeled data are sent to each participant of the vertical federated learning, so that each participant groups the labeled data at each participant based on the sample identifiers in each group of labeled data.

6. The model training method according to claim 5, characterized in that: In the case where the federated learning is horizontal federated learning, the labeled data of each party in the federated learning is grouped to obtain multiple groups of labeled data, including: Receive the data volume and / or label number of the labeled data sent by each participant of the horizontal federated learning; Based on the data amount and / or label number of the labeled data at each participant, the labeled data at each participant is grouped to obtain multiple groups of labeled data.

7. The model training method according to claim 6, characterized in that: The method further comprises: grouping the labeled data at each participant based on the amount of the labeled data at each participant, including: Determine a data volume target value based on the number of groups and the data volume of the labeled data at each participant, and determine a current demand volume of each group of labeled data based on the data volume target value and the allocated data volume of each group of labeled data; Traverse the amount of ungrouped data of the labeled data at each participant, and when traversing to the current participant, traverse each group of labeled data, and when the amount of ungrouped data at the current participant is less than or equal to the current demand of the traversed current group of labeled data, allocate all the unallocated labeled data at the current participant to the current group of labeled data; Traverse the ungrouped data amount of the labeled data at each participant again, and when traversing to the current participant, traverse each group of labeled data, and when the ungrouped data amount at the current participant is less than or equal to the current demand amount of the traversed current group of labeled data, allocate all the unallocated labeled data at the current participant to the current group of labeled data; otherwise, allocate part of the unallocated labeled data at the current participant to the current group of labeled data, so that the allocated data amount of the current labeled data reaches the data amount target value.

8. The model training method according to claim 4, characterized in that: The model training is performed based on multiple sets of labeled data to obtain multiple reference models, including: Cross-combining the multiple groups of labeled data to obtain multiple training sample sets, each of the training sample sets containing groups of labeled data with numbers different from the numbers of the training sample sets; Model training is performed based on each of the training sample sets to obtain a reference model corresponding to each training sample set.

9. A model training device, characterized in that: include: A determination unit, used to determine a global model to be trained and a reference model trained based on the labeled data; A labeling application unit, configured to perform reasoning on the labeled data based on the global model and the reference model, respectively, to obtain a global reasoning result and a reference reasoning result of the labeled data, and to determine a labeling deviation of the global model based on the global reasoning result and the reference reasoning result of the labeled data and a label of the labeled data; The unlabeled application unit is used to reason about the unlabeled data based on the global model and the reference model, respectively, to obtain the global reasoning result and the reference reasoning result of the unlabeled data, and to iterate the parameters of the global model based on the global reasoning result and the reference reasoning result of the unlabeled data and the labeled deviation, so as to complete the model training.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the model training method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the model training method as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the model training method as described in any one of claims 1 to 8.

Images