Auxiliary diagnosis method and system for mental illness based on multimodal data analysis model

Abstract

The present invention discloses a method and system for assisting diagnosis of mental illness based on a multimodal data analysis model. First, facial expression data, voice data and text data of users when they are interviewed are collected and preprocessed; then multimodal features are extracted, including features of facial expression data, voice data and text data after preprocessing; finally, the multimodal features are input into the multimodal data analysis model, and the classification results are output; the present invention designs a collection method that can simultaneously collect heterogeneous multimodal data of text, voice and facial expression of users in the multi-stage experimental process of "watching movies, reading poems and consulting with digital doctors"; heterogeneous multimodal data post-processing and heterogeneous multimodal mental illness analysis models are designed to assist doctors in intelligent diagnosis and improve the efficiency of doctors' consultation. The present invention develops an integrated system based on multimodal data collection and analysis, realizes intelligent auxiliary diagnosis of patients with common mental illnesses, and saves labor costs.

Images