Cross-modal image mode identification method for biomedical literature

Abstract

The invention relates to a cross-modal image mode identification method for the biomedical literature, belongs to the field of image identification and natural language processing and aims to solve across-modal image identification problem in the medical literature. The method is characterized in that an image and a text are processed through utilizing a pre-trained cross-modal composite image detection model, and whether the to-be-identified image is a composite image is determined; the image and the text are processed through utilizing a pre-trained cross-modal composite image multi-label classification model to output a biomedical pattern category of a composite image sub picture; a pre-trained cross-modal simple image pattern classification model is utilized to process the image and the text to output a biomedical pattern category of simple images. The method is advantaged in that image pattern identification tasks in the biomedical literature are effectively completed, resourcesin the general and biomedical fields are utilized to improve the identification performance, and labor and time cost is reduced.

Description

technical field [0001] The invention relates to the fields of image recognition and natural language processing, in particular to a cross-modal image pattern recognition method for biomedical documents. Background technique [0002] With the development of the Internet, the number of digital biomedical literature is increasing day by day. Users around the world can grasp the latest developments in their field and make new inventions or discoveries by searching literature. Schematic representation of digital medical imaging and medical data redrawing, as an important part of biomedical literature, plays an indispensable role in medical research and education. [0003] Using a large number of existing medical terminology resources, such as MeSH, IRMA, and RadLex, to classify medical concepts, and then combining ontology knowledge in the medical field and low-level visual features of images, the concepts are mapped to different biomedical image categories, and then a refined hi...

AI Technical Summary

Problems solved by technology

Third, the cost of data labeling and data characteristics in the biomedical field affect the scale and quality of training data, and limit the advantages of deep convolutional neural networks. However, there are large-scale (million-level) labeled There are also small-scale labeled data in the biomedical field. If the labeled data in the general field and the biomedical field are used comprehensively, it will bring a huge improvement to the image pattern recognition performance of biomedical literature.

[0006] However, the traditional image pattern recognition technology of biomedical literature, according to the experience of experts, manually fits the features, trains the classifier, and recognizes the biomedical pattern of the image

This method of feature engineering is highly dependent on dictionaries and rules, and some biomedical image patterns have significant differences and some are subtle. Faced with poor generalization performance of diverse samples, it is difficult to achieve better classification performance.

Images