Regularization-based social prejudice removing language model and application

Abstract

The invention discloses a regularization-based social prejudice removing language model and an application. The method comprises the following steps: (1) after cleaning a PTB corpus text library, screening words which may have social prejudice in the PTB corpus text library, and marking the words; (2) establishing a language model comprising three layers of LSTM networks, a full connection layer and a softmax layer; (3) training a language model by utilizing the PTB corpus text library, and taking the total loss Loss consisting of the loss Loss of the text generation task and the loss Loss ofthe de-social prejudice regularization item as the final loss during training; and (4) in each training stage, evaluating the social prejudgement effect of the language model according to the distribution state of the social prejudgement score of the prediction text output by the language model relative to the social prejudgement score of the PTB corpus text, and obtaining a final language model when the distribution state is satisfactory. The language model improves the fairness of prediction output.

Description

technical field [0001] The invention belongs to natural language processing models, and in particular relates to a regularization-based language model and its application without social prejudice. Background technique [0002] Artificial intelligence governance has become a topic of widespread concern in recent years, and the fairness of deep learning is the most critical issue in artificial intelligence governance. How to effectively deal with discriminatory biased data in training datasets is a major problem currently facing machine learning. Biased training data sets are generally considered to be one of the important factors affecting the fairness and justice of machine learning. Most machine learning models are trained on large labeled datasets. For example, in natural language processing, standard algorithms are trained on corpora containing billions of words. Researchers typically construct such datasets by scraping websites such as Google Images and Google News, u...

AI Technical Summary

Problems solved by technology

In an experimental setting, researchers use a test dataset for evaluation to verify the effectiveness of the algorithm, but the test set is usually a random subsample of the original training dataset and thus may contain the same bias

Such data and algorithm bias has become a growing problem

Images