Multi-modal mental health analysis method based on sand table image, text and video

Abstract

The invention discloses a multi-modal mental health analysis method based on sand table images, texts and videos. The method comprises the following steps: respectively obtaining sand table image, face image and text test results through a multi-modal mental health analysis system; obtaining a sand painting image feature vector, a facial expression feature vector and a text feature vector after feature extraction processing; and finally, dynamically adjusting the weight coefficient of each modal by adopting an attention mechanism, fusing by adopting a weighted average mode, performing reasoning calculation through an LSTM network, and finally inputting the result into a classifier to obtain a multi-modal mental health state prediction result. The invention belongs to the technical field of psychological health analysis, and particularly provides a multi-modal psychological health analysis method based on sand table images, texts and videos, which can avoid the limitation of a traditional psychological assessment scene, provides a multi-modal psychological sand table assessment analysis mode, integrates various test information and greatly improves the assessment accuracy.

Description

technical field [0001] The invention belongs to the technical field of mental health analysis, and specifically refers to a multimodal mental health analysis method based on sand table images, texts and videos. Background technique [0002] The initial sand table refers to the first sand table made by the sand table player, which can reflect the essence of the sand table player's psychological problems. Sandplay has been studied as a diagnostic tool by comparing sandplay between sick and healthy individuals. The existing sand table theory is based on the dimensions of sand table use, descriptive features, and sand table theme characteristics of the initial sand table to find out the characteristics of positive psychological symptoms in the sand table game, so as to achieve the application of the initial sand table in clinical diagnosis. [0003] With the explosive growth of Internet design media, more and more users publish what they see, hear and feel through social networ...

AI Technical Summary

Problems solved by technology

[0006] Due to the limitations of technical assessment capabilities, the psychological assessment techniques introduced above have certain bottlenecks in accuracy

For example, a patient with depression did not show positive symptoms in the sand table projection test, but he could be identified as positive in the video recognition through the expression features. This is due to the individual differences of the subjects, resulting in low reliability and validity of the evaluation tool.

[0007] The above three psychological assessment techniques cannot meet all the assessment needs due to the individual differences of the testees, so there is an urgent need for a technique that can integrate the three kinds of information for accurate assessment of psychological symptoms

Images