Distant Supervision for Biomedical Entity Relation Extraction

The patent applies preliminary action by using pre-trained language models (BERT, RoBERTa, BioBERT) that have been pre-trained on large-scale corpora before being applied to biomedical relation extraction. This pre-training phase performs the costly computation once, and the resulting models can be efficiently fine-tuned on smaller biomedical datasets, significantly reducing the time required for data labeling and model training while maintaining high extraction accuracy.

The patent uses copying by leveraging pre-trained language models that have learned general language patterns and representations from large corpora. These pre-trained models serve as templates that can be copied and adapted to biomedical domains through fine-tuning, avoiding the need to train models from scratch on limited biomedical data, thus reducing labeling time while preserving accuracy.

The patent applies preliminary action by performing pre-training on large-scale general corpora before domain-specific fine-tuning. This preliminary pre-training accumulates computational resources and learning patterns once, allowing the model to achieve high accuracy on biomedical relation extraction with much smaller domain-specific training data, thus reducing the quantity of required training data while maintaining accuracy.

The patent uses copying by adapting pre-trained language models to biomedical domains. The pre-trained models contain learned representations from large corpora that can be copied and reused, eliminating the need to retrain on all data from scratch. This allows high-accuracy relation extraction with reduced training data quantity by leveraging the copied knowledge from pre-trained models.

The patent merges the advantages of both supervised and distant supervision approaches. It combines pre-trained language models with distant supervision techniques, where the pre-trained models provide robust linguistic understanding and the distant supervision leverages external knowledge bases and automated annotation methods. This merging allows the system to achieve high accuracy while reducing manual labeling costs, as the pre-trained models can process large amounts of unlabelled data efficiently.

The patent uses intermediary by introducing pre-trained language models as mediators between raw biomedical text and relation extraction tasks. These intermediaries (pre-trained models) can process unlabelled data and generate predictions that can be used for training or direct inference, reducing the need for manual labeling while maintaining accuracy through the intermediary's pre-learned representations and reasoning capabilities.