Sample data processing method and device and multi-party model training system

Abstract

The embodiment of the invention provides a sample data processing method and device for multi-party model training. In the method, a first sample dataset is classified into a second sample dataset anda third sample dataset based on data tags of sample data, the second sample data in the second sample dataset having a unique data tag, and the third sample data in the third sample dataset having atleast two different data tags; performing model training by using the second sample data set to train a first model; performing data quality evaluation on the local data of each first member node byusing the first model; based on the data quality evaluation result of each first member node, performing label reconstruction on third sample data in a third sample data set, wherein the third sampledata after label reconstruction has a unique data label.

Description

technical field [0001] The embodiments of this specification generally relate to the field of artificial intelligence, and in particular, relate to a sample data processing method and device for multi-party model training, and a multi-party model training system. Background technique [0002] With the development of artificial intelligence technology, business models such as deep neural network (DNN) have been gradually applied to various business application scenarios, such as risk assessment, speech recognition, natural language processing, etc. The model structure of business models in different application scenarios is relatively fixed. In order to achieve better model performance, more data owners are required to provide more training sample data during model training. For example, when a business model is applied to medical, financial and other fields, different medical or financial institutions will collect different data samples. Once these data samples are used to ...

AI Technical Summary

Problems solved by technology

[0003] For joint training involving multiple data owners, different data owners have different data sources, different data collection methods, and various errors in the data aggregation process, so that for the same training model, the data quality of each data owner is also different. are not the same, and for data samples with the same data identifier (ID), different data owners will also generate different sample labels, so that there is a label conflict between the data samples collected by each data owner

Images