Multimode intelligent analysis method and system

Abstract

The invention relates to a multimode intelligent analysis method and a system. The disease description is multimode data, and the multimode data comprises character data, time sequence signal data and visual data; the method can be applied to the diagnosis navigation and diagnosis decision according to the disease situation; the method includes steps of converting visual data to character description through an image-text decoder; converting a time sequence signal to the text description through a signal character decoder; respectively converting the character data, the character description of the visual data and the time sequence signal data to a semantic vector taking a medical treatment mapping knowledge domain as an axis, wherein the medical treatment mapping knowledge domain describes the topological relation among the symptom, laboratorial index, disease, drug and operation; according to the semantic vector, a first convolution neutral network CNN model is applied to perform diagnosis navigation and diagnosis decision. According to the embodiment, the best diagnosis path and the diagnosis result with high accuracy can be acquired.

Description

technical field [0001] The invention relates to the technical field of information processing, in particular to a multimodal intelligent analysis method and system. Background technique [0002] With the development of science and technology, information processing technology is gradually applied to clinical diagnosis, which is called intelligent disease diagnosis. Multimodal intelligent analysis methods are often used in intelligent disease diagnosis, and intelligent disease diagnosis is naturally a multimodal machine learning problem. [0003] Multi-modal machine learning aims to build a machine learning model for a research goal, process and correlate heterogeneous data from multiple sources for that goal. There are mainly three types of data, text (Verbal), audio (Vocal), and visual (Visual). [0004] Intelligent disease diagnosis also needs to comprehensively consider multi-source heterogeneous data, including symptoms spoken by patients, signs measured by machines, l...

AI Technical Summary

Problems solved by technology

c. There are ambiguities in subjective descriptions and human clerical errors in the written records, and there are not only human errors in the signal sequence, but also the noise of the machine itself

d. The number of written records, signal sequences, and video pictures varies, with uneven distribution and inconsistent reliability

[0006] Due to the inability to find a suitable information representation method for multi-modal information representation (Representation) in the prior art, it is impossible to obtain the optimal diagnosis path and diagnosis results with a high accuracy rate

Images