Pathological classification method and system based on multi-modal deep learning

A technology of deep learning and classification method, which is applied in the fields of computer vision and image processing, and can solve the problem of low classification accuracy of benign and malignant breast cancer

Active Publication Date: 2019-10-01
INST OF COMPUTING TECH CHINESE ACAD OF SCI +1
View PDF5 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Aiming at the deficiencies of the above-mentioned prior art, the present invention proposes a breast cancer classification method based on multimodal deep learning, wh

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Pathological classification method and system based on multi-modal deep learning
  • Pathological classification method and system based on multi-modal deep learning
  • Pathological classification method and system based on multi-modal deep learning

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0029] This application proposes a data fusion method to simulate pathological diagnosis tasks. From the perspective of multi-modal data fusion, try to combine the pathological images in the electronic medical record EMR with structured data to further improve the accuracy of breast cancer diagnosis. This is also in line with the actual situation when pathologists read pathological images for diagnosis. When reading pathological images, pathologists will repeatedly refer to the relevant clinical structured information in the patient's EMR as a priori until the final diagnosis is made. Among them, 29 representative attributes were extracted from clinical electronic medical records through discussions with pathologists and review of medical literature related to breast cancer. These attributes are closely related to the diagnosis of breast cancer in medical theory, and these 29 attributes are all routine clinical indicators, which can be directly obtained from the database of th...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a pathological classification method and system based on multi-modal deep learning. The method comprises: extracting pre-selected attributes from the electronic medical recordsto serve as feature representation vectors of structural data, randomly discarding the feature representation vectors according to a preset proportion after being averagely amplified, and replacing the discarded parts with numbers 0 to serve as medical record feature vectors of the structural data in the electronic medical records; obtaining a histopathology image corresponding to the electronic medical record, performing global average pooling on the feature map of each convolutional layer of the convolutional neural network, and splicing the feature maps into a one-dimensional vector to serve as a rich image feature vector of the histopathology image; and splicing the image feature vector and the medical record feature vector together to obtain a multi-mode fusion vector, and inputting the multi-mode fusion vector into a full connection layer to obtain a binarized pathological classification result. The technical problem that the accuracy of pathological benign and malignant classification through single-mode feature representation is not high is solved.

Description

technical field [0001] The present invention relates to the field of computer vision and image processing in computer technology, in particular to a pathological classification method and system based on multimodal deep learning. Background technique [0002] Cancer is an important worldwide public health problem. Among all cancer types, breast cancer is the second most common cancer in women. Additionally, breast cancer has a very high mortality rate compared to other types of cancer. Despite the rapid development of medical science, pathological image analysis is still the most widely used method in breast cancer diagnosis. However, the dramatic increase in complexity and workload of histopathology images makes this task time-consuming, and its results are susceptible to pathologist subjectivity. In the face of this problem, the development of accurate breast cancer automatic diagnosis method is a very urgent need in this field. [0003] In recent years, deep learning ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/62G06K9/46G06N3/04G06N3/08G16H50/20G16H50/70
CPCG16H50/20G16H50/70G06N3/08G06V10/40G06N3/045G06F18/2411G06F18/253
Inventor 张法颜锐谭光明任菲刘志勇刘玉东张云峰
Owner INST OF COMPUTING TECH CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap