A method, system, device and medium for screening and evaluating mental health status

Abstract

The present invention discloses a method, system, device and medium for general screening and evaluation of mental health status, which relates to the field of mental health technology, including: based on the initial information of the user, a personalized psychological assessment questionnaire is customized through machine learning; based on the deep learning architecture combined with the convolutional neural network, an emotional cognitive neural network model is constructed to deeply analyze personal psychological data and form the analysis results of personal psychological and physiological indicators; combined with the evaluation feedback of the personalized psychological assessment questionnaire and the analysis results of personal psychological and physiological indicators, a comprehensive evaluation is performed through a random forest model to generate a personalized mental health status feedback report and improvement suggestions. The present invention uses dynamic questionnaires and multimodal data analysis, combined with deep learning and random forest models, to accurately evaluate mental health, generate a report containing customized improvement suggestions, and comprehensively improve the pertinence, accuracy and guidance of the evaluation.

Description

Technical Field

[0001] The present invention relates to the field of mental health technology, and in particular to a method, system, equipment and medium for screening and assessing mental health conditions. Background Art

[0002] With the rapid development of society and the accelerated pace of life, mental health issues are increasingly receiving widespread attention. Traditional mental health assessment methods mainly rely on clinical interviews and standardized psychological test questionnaires. However, these methods have the limitations of being highly subjective, time-consuming, and difficult to promote on a large scale. In recent years, the application of artificial intelligence technology in the field of mental health assessment has gradually emerged, especially the development of machine learning and deep learning, which has provided new possibilities for rapid screening and accurate assessment of mental health conditions. For example, emotion recognition technology based on machine learning can automatically analyze an individual's voice and facial expressions, while deep learning can extract deep features from complex data to build a more accurate mental health assessment model. In addition, the rise of multimodal data fusion technology has made it possible to comprehensively assess an individual's mental health from multiple perspectives, improving the comprehensiveness and accuracy of the assessment.

[0003] Although the application of artificial intelligence in the field of mental health assessment has made significant progress, there are still some shortcomings in existing technologies. First, most mental health assessment systems lack personalized design and fail to fully consider individual differences, resulting in limited applicability and accuracy of assessment results. Second, existing assessment methods often focus on a single data source, such as relying solely on questionnaires or biosignal monitoring, ignoring the potential of multimodal data fusion, which limits the comprehensiveness and reliability of the assessment system. Third, existing technologies are often not accurate when dealing with complex psychological states, such as emotion recognition and cognitive bias analysis, especially in distinguishing subtle psychological differences. Finally, most assessment systems lack targeted improvement suggestions when generating feedback reports, making it difficult to meet the needs of individualized mental health interventions. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method, system, device and medium for general screening and assessment of mental health conditions to solve the deficiencies of existing assessment systems in terms of personalization, comprehensiveness, accuracy and intervention guidance.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In the first aspect, an embodiment of the present invention provides a method for screening and evaluating mental health conditions, which includes customizing a personalized psychological assessment questionnaire through machine learning based on the user's initial information; collecting the user's voice and facial video for emotion recognition while the user is completing the questionnaire, and constructing comprehensive personal psychological data through multimodal data fusion; constructing an emotional cognitive neural network model based on a deep learning architecture combined with a convolutional neural network to deeply analyze personal psychological data and form analysis results of personal psychological and physiological indicators; combining the evaluation feedback of the personalized psychological assessment questionnaire and the analysis results of personal psychological and physiological indicators, a comprehensive evaluation is performed through a random forest model to generate a personalized mental health status feedback report and improvement suggestions.

[0008] As a preferred embodiment of the method for screening and assessing mental health conditions of the present invention, the method of customizing a personalized psychological assessment questionnaire based on the initial information of the user through machine learning comprises the following specific steps:

[0009] Based on the mental health database and research, basic information about users’ age, gender, occupation, past mental health status, life stress level and social support is collected as sample data;

[0010] The missing values ​​of the sample data are processed by filling in the mean, filling in the median and interpolation to form the original sample data;

[0011] Based on machine learning algorithms and automated data exploration, the principal components of the original sample data are extracted through factor analysis;

[0012] Use recursive feature elimination to preliminarily screen the principal components as the user's initial information;

[0013] Quantify the importance of each principal component for model prediction through random forest classifier;

[0014] Based on the importance scores of each principal component quantified by the random forest classifier, a weight is assigned to each principal component;

[0015] A personalized psychological assessment questionnaire is dynamically constructed based on the user's initial information and weighted principal components.

[0016] As a preferred solution of the method for screening and assessing mental health conditions of the present invention, the user's voice and facial video are collected for emotion recognition while the user is completing the questionnaire. The specific steps are as follows:

[0017] While the user is completing the questionnaire, the user's voice file and facial video file are recorded synchronously, and the timestamps of the two are aligned;

[0018] Perform wavelet denoising on the synchronously recorded user voice file, and use Mel frequency cepstral coefficients to extract the voice feature sequence to generate an output voice feature sequence;

[0019] Use the long short-term memory model to classify the emotion of the speech feature sequence and generate speech emotion features;

[0020] Based on the facial video file, the OpenFace toolkit is used to perform face detection and alignment, and extract facial feature sequences from it;

[0021] Through the three-dimensional convolutional neural network model, the facial feature sequence is subjected to high-level emotion classification to generate facial emotion features;

[0022] The speech emotion features and facial emotion features are weightedly fused using the weight distribution principle, and emotion recognition is performed through comprehensive emotion judgment.

[0023] As a preferred solution of the method for screening and assessing mental health conditions of the present invention, the steps of constructing comprehensive personal psychological data through multimodal data fusion are as follows:

[0024] Multimodal data fusion includes personalized psychological assessment questionnaire data, facial emotion feature data, and voice emotion feature data;

[0025] The Likert scale was used to encode the personalized psychological assessment questionnaire data and quantify them into questionnaire score values;

[0026] Performing denoising and feature extraction preprocessing on the facial emotion feature data and the speech emotion feature data to generate speech emotion features and facial emotion features;

[0027] Through feature-level fusion, the quantified questionnaire score values, voice emotion features, and facial emotion features are spliced ​​to form a comprehensive multimodal feature vector;

[0028] The comprehensive multimodal feature vectors are combined with emotion recognition data to form a comprehensive personal psychological dataset containing self-report, voice emotion and facial emotion.

[0029] As a preferred solution of the method for screening and assessing mental health conditions described in the present invention, the method is based on a deep learning architecture combined with a convolutional neural network to construct an emotional cognitive neural network model to deeply analyze personal psychological data and form analysis results of personal psychological and physiological indicators. The specific steps are as follows:

[0030] Convert each modal data in the comprehensive personal psychological data set into a unified feature representation. Assume that the questionnaire feature is Q, the voice emotion feature is S, and the facial emotion feature is F. Then the multimodal feature vector X is expressed as: ;

[0031] Apply kernel principal component analysis with adaptive weight allocation to the multimodal feature vector X to perform feature enhancement and dimensionality reduction to obtain the enhanced feature vector The expression is:

[0032] ;

[0033] Among them, Φ is the kernel function mapping, is the dimension reduction weight matrix;

[0034] The feature vector will be enhanced Input into a multi-layer fusion and convolutional network, combined with the sentiment fusion function and nonlinear transformation function The expression of the generated fused feature vector Z is:

[0035] ;

[0036] Among them, α i is the attention weight of the i-th modality

[0037] Input the fused feature vector Z into the one-dimensional convolution layer and calculate the output value of the l-th convolution layer , the expression is:

[0038] ;

[0039] in, is the corrected linear unit, is the weight matrix of the l-th layer convolution, is the bias term of the lth layer;

[0040] The output of the convolutional network is mapped to the psychological and physiological index space through the fully connected layer to quantitatively predict the individual's psychological and physiological state. The expression is:

[0041] ;

[0042] in, is the predicted psychological and physiological index vector, is the output layer weight, is the output layer bias, Represents the output feature vector of the convolutional network;

[0043] Based on the value range of the predicted psychological and physiological index vector Y, the psychological and physiological index type is output to form the analysis result of the personal psychological and physiological index.

[0044] As a preferred embodiment of the method for screening and assessing mental health conditions of the present invention, the evaluation feedback of the personalized psychological assessment questionnaire and the analysis results of the individual psychological and physiological indicators are combined to conduct a comprehensive assessment through a random forest model, and the specific steps are as follows:

[0045] Based on questionnaire features Q, voice emotion features S and facial emotion features F, construct the emotion influencing factor E i The expression is:

[0046] ;

[0047] in, is the baseline importance score of the ith modality, σ i is the standard deviation of the i-th mode data;

[0048] The matrix R is thresholded based on the Pearson correlation coefficient, which is used to quantify the relationship between different psychological and physiological indicators, and a sparse correlation matrix R is constructed. se , and through singular value decomposition SVD, the low-rank representation matrix U is obtained, expressed as:

[0049] ;

[0050] Combined with emotional influence factor E i and the sparse correlation matrix R se , and integrate an indicator function 1y t =yThe expression of constructing a multimodal sentiment weighted random forest model is:

[0051] ;

[0052] Where P(y|x) is the probability of predicting that all modal features x of a given input belong to the psychological and physiological state y, T is the total number of decision trees in the random forest, and β is the modulating emotion influence factor E. i And the influence of the cross-modal feature combination guided by the low-rank representation matrix U in the calculation of P(y|x).

[0053] As a preferred embodiment of the method for screening and assessing mental health conditions of the present invention, the specific steps of generating a personalized mental health feedback report and improvement suggestions are as follows:

[0054] Based on the predicted probability P(y|x) of the multimodal emotion-weighted random forest model, the expected probability of each psychological and physiological state y is calculated to construct a comprehensive health score , the expression is:

[0055] ;

[0056] Among them, Ih (y) is the overall impact index of psychological and physiological state y on mental health;

[0057] According to the feature importance and sentiment influence factor E of the random forest model i , identify the key psychophysiological indicators that have the greatest impact on current mental health status;

[0058] Provide a set of personalized improvement strategies based on the identified key psychophysiological indicators.

[0059] In a second aspect, the present invention provides a mental health screening and assessment system, including a questionnaire customization module, an emotion recognition module, a deep analysis module, a mental health assessment module and a module for generating personalized suggestion reports; the questionnaire customization module is used to customize a personalized psychological assessment questionnaire based on the user's initial information through machine learning; the emotion recognition module is used to collect the user's voice and facial video for emotion recognition while the user is completing the questionnaire, and to construct comprehensive personal psychological data through multimodal data fusion; the deep analysis module is used to construct an emotion cognitive neural network model based on a deep learning architecture combined with a convolutional neural network to deeply analyze personal psychological data and form analysis results of personal psychological and physiological indicators; the mental health assessment module is used to combine the evaluation feedback of the personalized psychological assessment questionnaire and the analysis results of personal psychological and physiological indicators, and perform a comprehensive assessment through a random forest model; the module for generating personalized suggestion reports is used to generate personalized mental health feedback reports and improvement suggestions.

[0060] In a third aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the method for general screening and assessment of mental health conditions as described in the first aspect of the present invention is implemented.

[0061] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the method for general screening and assessment of mental health conditions as described in the first aspect of the present invention is implemented.

[0062] The beneficial effects of the present invention are as follows: the present invention dynamically constructs a personalized psychological assessment questionnaire by combining the user's initial information and machine learning technology, thereby ensuring the pertinence and effectiveness of the assessment process; at the same time, it collects user voice and facial video data, and uses multimodal data fusion technology to achieve deep analysis of emotion recognition and psychological and physiological indicators, thereby constructing comprehensive personal psychological data; in addition, through the comprehensive application of deep learning architecture and random forest model, the present invention can not only accurately assess the individual's mental health status, but also generate a feedback report containing personalized improvement suggestions, providing users with customized mental health guidance. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0064] Figure 1 This is a flow chart of the method for general screening and assessment of mental health status in Example 1.

[0065] Figure 2 This is a flow chart for constructing comprehensive personal psychological data in Example 1. DETAILED DESCRIPTION

[0066] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0067] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0068] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0069] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a method for screening and assessing mental health conditions, comprising the following steps:

[0070] S1. Based on the user's initial information, a personalized psychological assessment questionnaire is customized through machine learning.

[0071] Furthermore, based on the mental health database and research, a large amount of basic information on users’ age, gender, occupation, past mental health status, life stress level, and social support is collected as sample data;

[0072] The missing values ​​of the sample data are processed by filling in the mean, filling in the median and interpolation to ensure the integrity of the data and form the original sample data;

[0073] Based on machine learning algorithms and automated data exploration, the principal components in the original sample data are extracted through factor analysis. Factor analysis is a statistical method that aims to simplify complex data structures and reveal the basic structure of data by reducing the number of variables. The expression is:

[0074] ;

[0075] Among them, F represents the observed variable matrix, A is the factor loading matrix, Λ represents the factor score matrix, and G is the residual matrix of a specific variable;

[0076] Use recursive feature elimination to preliminarily screen the principal components as the user's initial information;

[0077] Furthermore, the importance of each principal component for model prediction is quantified by the random forest classifier, and the expression is:

[0078] ;

[0079] Among them, I m is the importance score of the mth feature, E is the expected value, and △Err is the increase in error caused by the random permutation of feature m;

[0080] Based on the importance scores of each principal component quantified by the random forest classifier, a weight is assigned to each principal component. The higher the weight, the greater the impact of the feature on the assessment of mental health status.

[0081] Based on the user's initial information and weighted principal components, the process of dynamically constructing a personalized psychological assessment questionnaire involves intelligent selection and sorting of questionnaire questions to ensure that the questionnaire content covers key psychological assessment areas and can be adjusted to the user's specific background and needs. For example, for users with higher stress levels, the questionnaire may focus more on assessing coping mechanisms and stress management skills.

[0082] It should also be noted that by deeply mining the value of users' initial information and combining it with advanced data processing and machine learning technologies, we have achieved personalized customization of mental health assessment questionnaires, which not only improves the pertinence and efficiency of the assessment, but also provides a solid foundation for subsequent in-depth analysis and personalized recommendations. At the same time, it has significantly improved the sophistication of data processing, the effectiveness of feature screening, and the flexibility of questionnaire customization, bringing innovative breakthroughs to the field of mental health assessment.

[0083] S2. While users are completing the questionnaire, their voice and facial video are collected for emotion recognition, and comprehensive personal psychological data is constructed through multimodal data fusion.

[0084] Furthermore, while the user is taking the questionnaire, the user's voice file and facial video file are recorded synchronously, and the timestamps of the two are aligned to achieve accurate association of cross-modal data;

[0085] Perform wavelet denoising on the synchronously recorded user voice files, extract voice feature sequences using Mel Frequency Cepstral Coefficients (MFCCs), and output voice feature sequences;

[0086] Preferably, wavelet transform is a time-frequency analysis method that can effectively remove noise and retain useful information in the signal; Mel frequency cepstral coefficients (MFCCs) are an important feature extraction method widely used in speech recognition and audio processing. It simulates the perception characteristics of the human auditory system to sound, and is particularly effective for processing speech signals;

[0087] Use the long short-term memory model (LSTM) to classify the feature sequence and generate speech emotion features;

[0088] It should be noted that the implementation process of "generating speech emotion features" includes the following steps: first, extract useful features from the user's voice, such as Mel frequency cepstral coefficients (MFCCs), zero crossing rate, energy, pitch, etc. These features can capture the emotional color of the voice, standardize or normalize the extracted features so that they have the same scale, and avoid some features dominating the model training due to their large numerical range. Organize the extracted features into a sequence form. Each frame of speech corresponds to a feature vector, forming a time series, which is the input form of the LSTM model; secondly, construct a long short-term memory model (LSTM). The LSTM layer is the core of the model. It contains multiple LSTM units. Each unit can remember long-term dependent information and control the flow of information through a gating mechanism. The input layer of the model needs to match the dimension of the preprocessed feature sequence. The output layer of the model is usually a fully connected layer used to connect the LSTM The output of the layer is converted into the probability of emotion category, which can be a binary or multi-classification output, depending on the requirements of the emotion classification task. An appropriate loss function, such as the cross entropy loss function, is selected to guide the optimization direction of the model, and an optimizer, such as Adam, SGD, is defined to update the model parameters. Next, the long short-term memory model (LSTM) is trained, the data set is divided into a training set, a validation set, and a test set, the model is trained using the training set, the model parameters are updated through the back propagation algorithm, and the loss function is minimized. It may be necessary to iteratively adjust the number of LSTM units, the learning rate and other hyperparameters for multiple times to achieve the best performance, and the model performance is evaluated on the validation set to prevent overfitting. Finally, the feature sequence is classified through the trained long short-term memory model (LSTM), and the probability distribution of the emotion category or emotion tendency is output. According to the model output, specific speech emotion features are generated, such as emotion labels such as happiness, sadness, and anger.

[0089] Furthermore, based on the facial video files, the OpenFace toolkit is used to perform face detection and alignment to ensure accurate positioning of facial key points and extract facial feature sequences (such as expression intensity, eye direction, etc.) from them;

[0090] Through the 3D convolutional neural network model, the facial feature sequence is subjected to high-level emotion classification (in high-level emotion classification, facial detection algorithms such as Haar features, HOG+SVM or deep learning methods are used to locate the facial area in the image) to generate facial emotion features;

[0091] The speech emotion features and facial emotion features are weightedly fused using the weight distribution principle (the weight is based on the evaluation results of the feature importance in the previous step), and emotion recognition is performed through comprehensive emotion judgment;

[0092] Multimodal data fusion includes personalized psychological assessment questionnaire data, facial emotion feature data, and voice emotion feature data;

[0093] The Likert scale was used to encode the personalized psychological assessment questionnaire data and quantify them into questionnaire score values;

[0094] Preferably, the "Likert scale", also known as the Likert scale, was proposed by American social psychologist Rensis Likert in 1932 to measure people's attitudes or opinions on specific topics. This scale is a common evaluation tool, often used in social science research, market research and psychological evaluation, and is one of the most widely used scales in survey research;

[0095] Performing denoising and feature extraction preprocessing on the facial emotion feature data and the speech emotion feature data to generate speech emotion features and facial emotion features;

[0096] Through feature-level fusion, the quantified questionnaire score values, voice emotion features, and facial emotion features are spliced ​​to form a comprehensive multimodal feature vector;

[0097] Combining comprehensive multimodal feature vectors with emotion recognition data, a comprehensive personal psychological data set including self-report, voice emotion and facial emotion is formed. This data set not only includes traditional questionnaire assessment results, but also incorporates real-time emotional state information, providing a more comprehensive and in-depth perspective for subsequent mental health status assessment and intervention;

[0098] It should also be noted that by collecting and analyzing users' voice and facial video data in real time and combining it with the results of personalized psychological assessment questionnaires, deep fusion of multimodal data is achieved, providing multi-dimensional and high-precision support for the assessment of personal psychological conditions. This solution has significant improvements in the real-time nature of data collection, the accuracy of feature extraction, and the comprehensiveness of data fusion, opening up a new path for intelligent assessment in the field of mental health.

[0099] S3. Based on the deep learning architecture combined with convolutional neural network, an emotional cognitive neural network model is constructed to deeply analyze personal psychological data and form analysis results of personal psychological and physiological indicators.

[0100] Furthermore, the modal data in the comprehensive personal psychological data set are converted into a unified feature representation. Let the questionnaire feature be Q, the voice emotion feature be S, and the facial emotion feature be F. Then the multimodal feature vector X is expressed as: ;

[0101] Apply kernel principal component analysis with adaptive weight allocation to the multimodal feature vector X to perform feature enhancement and dimensionality reduction to obtain the enhanced feature vector The expression is:

[0102] ;

[0103] Among them, Φ is the kernel function mapping, is the dimension reduction weight matrix;

[0104] Preferably, kernel PCA can be used to find the linear separability of data in a higher-dimensional feature space, while reducing the data dimension and improving the training efficiency of subsequent models.

[0105] The feature vector will be enhanced Input into a multi-layer fusion and convolutional network and combined with the sentiment fusion function F E and nonlinear transformation function , the expression of the generated fused feature vector Z is:

[0106] ;

[0107] Among them, α i is the attention weight of the i-th modality. Through the attention mechanism, the model can automatically learn and assign weights of different modal features, so that important features can be highlighted in the fusion process;

[0108] Input the fused feature Z into the one-dimensional convolution layer and calculate the output value of the l-th convolution layer , the expression is:

[0109] ;

[0110] in, is the corrected linear unit, is the weight matrix of the l-th layer convolution, is the bias term of the lth layer;

[0111] Preferably, the one-dimensional convolutional layer can capture the local correlation between features and further enhance the expressiveness of the features;

[0112] Furthermore, the output of the convolutional network is mapped to the psychological and physiological index space through the fully connected layer to quantitatively predict the individual's psychological and physiological state. The expression is:

[0113] ;

[0114] in, is the predicted psychological and physiological index vector, is the output layer weight, is the output layer bias, Represents the output feature vector of the convolutional network, which represents the features extracted after the convolution operation;

[0115] Preferably, the fully connected layer maps the feature vector to the output space, completing the direct prediction from the feature to the psychophysiological index;

[0116] Based on the value range of the predicted psychophysiological indicator vector Y, the psychophysiological indicator type is output to form the analysis result of the personal psychophysiological indicator. This step converts the continuous predicted value into a specific indicator type for easy understanding and application.

[0117] It should be noted that the “value range of the psychological and physiological indicator vector Y” means that for a continuous variable, each value in Y will fall within a predefined value range. For example, the range of anxiety level may be 0 to 100, where 0 represents no anxiety and 100 represents extreme anxiety. The model will map each indicator to its corresponding value range based on the value of Y, and then determine the type or severity of the indicator based on this value range, which is used to form the analysis results of individual psychological and physiological indicators and provide personalized mental health assessment for individuals;

[0118] It should also be noted that through multimodal data fusion and deep analysis under the deep learning framework, an accurate assessment of an individual's psychological and physiological state is achieved; not only does it improve the accuracy and efficiency of the assessment, but it also provides strong data support for personalized intervention in mental health; this solution is feasible and fully integrates the current cutting-edge technologies in the field of deep learning, opening up new avenues for research and practice in the field of mental health.

[0119] S4. Combine the evaluation feedback from the personalized psychological assessment questionnaire and the analysis results of personal psychological and physiological indicators, conduct a comprehensive evaluation through the random forest model, and generate a personalized mental health status feedback report and improvement suggestions.

[0120] Furthermore, based on the questionnaire features Q, voice emotion features S and facial emotion features F, the emotion influencing factor E is constructed. i The expression is:

[0121] ;

[0122] in, is the baseline importance score of the ith modality, σ i is the standard deviation of the i-th mode data;

[0123] It should be noted that the emotional impact factor E i It reflects the relative importance and stability of each modality in emotion recognition and provides a weighted basis for the construction of subsequent models;

[0124] Based on the Pearson correlation coefficient, the matrix R is thresholded to construct a sparse correlation matrix Rse, and the low-rank representation matrix U is obtained through singular value decomposition SVD, which is expressed as:

[0125] ;

[0126] Preferably, the Pearson correlation coefficient is a statistic used to measure the degree of linear correlation between two variables. In the context of mental health assessment, it can be used to quantify the relationship between different psychophysiological indicators, such as the association between anxiety levels and heart rate variability; singular value decomposition SVD is a powerful linear algebra technique used to project high-dimensional data into a low-dimensional space while preserving the original information as much as possible. In mental health assessment, applying SVD to the sparse correlation matrix Rse can obtain a low-rank representation matrix U. This process essentially reduces the dimensionality while retaining the main trends and patterns in the data; the construction of the sparse correlation matrix Rse helps to remove redundant information, while the low-rank representation matrix U further compresses the data dimension and retains key covariation information;

[0127] Furthermore, the emotional influence factor Ei and the sparse correlation matrix Rse are combined, and an indicator function 1yt=y is integrated to construct a multimodal emotional weighted random forest model expression:

[0128] ;

[0129] Where P(y|x) is the probability of predicting that all modal features x of a given input belong to the psychological and physiological state y, T is the total number of decision trees in the random forest, and β is the adjustment emotion weight E iand the influence of the cross-modal feature combination guided by the U matrix in the calculation of P(y|x);

[0130] Based on the predicted probability P(y|x) of the multimodal emotion weighted random forest model, the expected probability of each psychological and physiological state y is calculated to construct a comprehensive health score Se hh The expression is:

[0131] ;

[0132] Among them, I h (y) is the overall impact index of state y on mental health. This score quantifies the mental health status of an individual and provides a quantitative basis for subsequent personalized recommendations;

[0133] According to the feature importance and emotional influence factor Ei of the random forest model, the key psychophysiological indicators that have the greatest impact on the current mental health state are identified;

[0134] Furthermore, a set of personalized improvement strategies is provided for the identified key psychological and physiological indicators, where the "extracted key psychological and physiological indicators" specifically refer to the indicators that have a significant impact on the individual's mental health status, which are analyzed from multimodal data. These indicators are derived from a comprehensive analysis of questionnaire characteristics, voice emotion characteristics, facial emotion characteristics and other data;

[0135] It should be noted that these indicators are identified as factors that have a decisive impact on mental health status through feature importance analysis of the random forest model. During the evaluation process, the system will combine the questionnaire characteristics and the analysis results of personal psychophysiological indicators to determine which indicators have the greatest impact on the current mental health status, so as to provide individuals with targeted improvement strategies. For example, if large fluctuations in emotional state are considered key indicators, then emotional regulation techniques and relaxation exercises will be included in the improvement suggestions. If cognitive bias is obvious, cognitive reconstruction exercises will be recommended first. In response to life pressure and insufficient social support, suggestions may be given to adjust the balance between work and rest and increase social interaction. Through such personalized analysis, improvement strategies can more effectively target the specific needs of individuals and improve the effectiveness of mental health interventions.

[0136] When the affect factor indicates that emotion is the primary factor, specific emotion regulation techniques, relaxation exercises, or professional counseling resources are recommended based on voice and facial affect characteristics;

[0137] When the questionnaire features indicate cognitive bias, cognitive restructuring exercises are proposed to guide users on how to identify and challenge negative thinking patterns;

[0138] Provide suggestions for adjusting work-rest balance and enhancing social interaction based on the user's life stress level and social support;

[0139] When heart rate variability is low, physiological indicators are abnormal, and breathing training and aerobic exercise are recommended to improve the function of the autonomic nervous system;

[0140] Integrate analysis and suggestions for improvement strategies to generate personalized mental health feedback reports and improvement suggestions. This report not only details the individual's mental health status, but also provides specific and feasible improvement measures to help users gradually improve their mental health.

[0141] It should be noted that through the organic combination of deep learning and random forest models, accurate assessment of individual mental health conditions and generation of personalized intervention recommendations are achieved, providing users with a comprehensive, personalized and highly operational health management plan.

[0142] The present embodiment also provides a mental health screening and assessment system, including a questionnaire customization module, an emotion recognition module, a deep analysis module, a mental health assessment module and a module for generating personalized suggestion reports; the questionnaire customization module is used to customize a personalized psychological assessment questionnaire based on the user's initial information through machine learning; the emotion recognition module is used to collect the user's voice and facial video for emotion recognition while the user is completing the questionnaire, and to construct comprehensive personal psychological data through multimodal data fusion; the deep analysis module is used to build an emotion cognitive neural network model based on a deep learning architecture combined with a convolutional neural network to deeply analyze personal psychological data and form analysis results of personal psychological and physiological indicators; the mental health assessment module is used to combine the evaluation feedback of the personalized psychological assessment questionnaire and the analysis results of personal psychological and physiological indicators, and perform a comprehensive assessment through a random forest model; the personalized suggestion report generation module is used to generate a personalized mental health feedback report and improvement suggestions.

[0143] This embodiment also provides a computer device suitable for the method of general screening and assessment of mental health conditions, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the method of general screening and assessment of mental health conditions proposed in the above embodiment.

[0144] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0145] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for implementing the general screening and assessment of mental health status proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination of them, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0146] In summary, the present invention innovates the mental health assessment process by integrating machine learning and deep learning technologies, customizes personalized questionnaires through algorithms, integrates multimodal data, uses deep learning models to analyze psychological and physiological indicators, and combines random forest evaluation to produce personalized health recommendations, which significantly improves the personalization of assessments, data processing accuracy and efficiency, and promotes the intelligence of mental health assessments.

[0147] Example 2, referring to Table 1, is the second example of the present invention. To further verify the advancement of the present invention, experimental simulation data of the general screening and evaluation method of mental health status are provided.

[0148] 100 volunteers were recruited from different backgrounds and divided into four groups, Subject A, Subject B, Subject C, and Subject D, for the experiment to ensure the diversity and representativeness of the sample. The age distribution of these volunteers ranged from 18 to 65 years old, covering different genders, occupations, and mental health histories.

[0149] First, we used machine learning algorithms to conduct an in-depth analysis of the mental health database and extract key features. We determined the principal components of age, gender, and occupation by performing factor analysis and recursive feature elimination on the sample data, and quantified their importance through a random forest classifier; based on this, we dynamically constructed a personalized psychological assessment questionnaire for each volunteer.

[0150] Secondly, while the volunteers completed the questionnaire, the system recorded their voice and facial videos, used wavelet denoising and Mel-frequency cepstral coefficients to extract voice features, used the OpenFace toolkit to extract facial features, and generated facial emotion features and voice emotion features through a three-dimensional convolutional neural network model and a long short-term memory model, respectively.

[0151] Next, the questionnaire features, voice emotion features and facial emotion features are fused at the feature level to form a comprehensive multimodal feature vector; then, the deep learning architecture is combined with the convolutional neural network to construct an emotion cognitive neural network model to deeply analyze the personal psychological data and form the analysis results of psychological and physiological indicators.

[0152] Finally, a personalized mental health status feedback report and improvement suggestions were generated by combining the evaluation feedback of the personalized psychological assessment questionnaire and the analysis results of personal psychophysiological indicators through the random forest model.

[0153] The details are shown in Table 1 below:

[0154] Table 1 Experimental record table

[0155]

[0156] Through the data analysis of the above table, it can be clearly seen that the effectiveness and superiority of the method of the present invention. Compared with traditional questionnaires, personalized questionnaires not only improve the willingness of participants to answer, but also through multimodal data fusion, the accuracy of emotion recognition is increased by about 10% on average, which shows that multimodal data fusion effectively improves the accuracy of emotion recognition; the emotional cognitive neural network model based on the deep learning architecture greatly reduces the prediction error of psychological and physiological indicators, with an average error of only 0.035, which is much lower than the 0.12 of the traditional evaluation method; this proves that the method of the present invention has significant advantages in accurately evaluating the individual's mental state; the comprehensive evaluation results show that the personalized mental health feedback report generated by the new method has a score that is closer to the actual mental health status, reflecting the comprehensiveness and accuracy of the evaluation, and the average score is increased to 84 points, while the traditional evaluation method is only about 70 points.

[0157] Through personalized questionnaires, multimodal data fusion, deep learning analysis and comprehensive evaluation models, the present invention achieves accurate assessment of individual mental health status and generation of personalized improvement suggestions, showing significant innovation and advantages in emotion recognition accuracy, prediction accuracy of psychological and physiological indicators and overall mental health scores.

[0158] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for screening and assessing mental health, characterized in that: include, Based on the user's initial information, a personalized psychological assessment questionnaire is customized through machine learning; While users are completing the questionnaire, their voice and facial video are collected for emotion recognition, and comprehensive personal psychological data is constructed through multimodal data fusion; Based on the deep learning architecture combined with convolutional neural network, an emotional cognitive neural network model is constructed to deeply analyze personal psychological data and form analysis results of personal psychological and physiological indicators; Combining the feedback from the personalized psychological assessment questionnaire and the analysis results of personal psychological and physiological indicators, a comprehensive assessment is conducted through the random forest model to generate a personalized mental health feedback report and improvement suggestions; The specific steps of customizing a personalized psychological assessment questionnaire based on the user's initial information through machine learning are as follows: Based on the mental health database and research, basic information about users’ age, gender, occupation, past mental health status, life stress level and social support is collected as sample data; The missing values ​​of the sample data are processed by filling in the mean, filling in the median and interpolation to form the original sample data; Based on machine learning algorithms and automated data exploration, the principal components of the original sample data are extracted through factor analysis; Use recursive feature elimination to preliminarily screen the principal components as the user's initial information; Quantify the importance of each principal component for model prediction through random forest classifier; Based on the importance scores of each principal component quantified by the random forest classifier, a weight is assigned to each principal component; Dynamically construct a personalized psychological assessment questionnaire based on the user's initial information and weighted principal components; The deep learning architecture combined with the convolutional neural network is used to build an emotional cognitive neural network model to deeply analyze personal psychological data and form analysis results of personal psychological and physiological indicators. The specific steps are: Convert each modal data in the comprehensive personal psychological data set into a unified feature representation. Assume that the questionnaire feature is Q, the voice emotion feature is S, and the facial emotion feature is F. Then the multimodal feature vector X is expressed as: ; Apply kernel principal component analysis with adaptive weight allocation to the multimodal feature vector X to perform feature enhancement and dimensionality reduction to obtain the enhanced feature vector The expression is: ； Among them, Φ is the kernel function mapping, is the dimension reduction weight matrix; The feature vector will be enhanced Input into a multi-layer fusion and convolutional network, combined with the sentiment fusion function and nonlinear transformation function The expression of the generated fused feature vector Z is: ； Among them, α i is the attention weight of the i-th modality; Input the fused feature vector Z into the one-dimensional convolution layer and calculate the output value of the l-th convolution layer , the expression is: ； in, is the corrected linear unit, is the weight matrix of the l-th layer convolution, is the bias term of the lth layer; The output of the convolutional network is mapped to the psychological and physiological index space through the fully connected layer to quantitatively predict the individual's psychological and physiological state. The expression is: ； in, is the predicted psychological and physiological index vector, is the output layer weight, is the output layer bias, Represents the output feature vector of the convolutional network; Outputting the psychological and physiological index type based on the value range of the predicted psychological and physiological index vector Y to form the analysis result of the personal psychological and physiological index; The evaluation feedback of the personalized psychological assessment questionnaire and the analysis results of personal psychological and physiological indicators are combined to conduct a comprehensive evaluation through a random forest model. The specific steps are as follows: Based on questionnaire features Q, voice emotion features S and facial emotion features F, construct the emotion influencing factor E i The expression is: ； in, is the baseline importance score of the ith modality, σ i is the standard deviation of the i-th mode data; The matrix R is thresholded based on the Pearson correlation coefficient, which is used to quantify the relationship between different psychological and physiological indicators, and a sparse correlation matrix R is constructed. se , and through singular value decomposition SVD, the low-rank representation matrix U is obtained, expressed as: ； Combined with emotional influence factor E i and the sparse correlation matrix R se , and integrate an indicator function 1y t =yThe expression of constructing a multimodal sentiment weighted random forest model is: ； Where P(y|x) is the probability of predicting that all modal features x of a given input belong to the psychological and physiological state y, T is the total number of decision trees in the random forest, and β is the modulating emotion influence factor E. i And the influence of the cross-modal feature combination guided by the low-rank representation matrix U in the calculation of P(y|x).

2. The method for screening and assessing mental health conditions according to claim 1, characterized in that: While the user is completing the questionnaire, the user's voice and facial video are collected for emotion recognition. The specific steps are as follows: While the user is completing the questionnaire, the user's voice file and facial video file are recorded synchronously, and the timestamps of the two are aligned; Perform wavelet denoising on the synchronously recorded user voice file, and use Mel frequency cepstral coefficients to extract the voice feature sequence to generate an output voice feature sequence; Use the long short-term memory model to classify the emotion of the speech feature sequence and generate speech emotion features; Based on the facial video file, the OpenFace toolkit is used to perform face detection and alignment, and extract facial feature sequences from it; Through the three-dimensional convolutional neural network model, the facial feature sequence is subjected to high-level emotion classification to generate facial emotion features; The speech emotion features and facial emotion features are weightedly fused using the weight distribution principle, and emotion recognition is performed through comprehensive emotion judgment.

3. The method for screening and assessing mental health conditions as claimed in claim 2, characterized in that: The specific steps of constructing comprehensive personal psychological data through multimodal data fusion are as follows: Multimodal data fusion includes personalized psychological assessment questionnaire data, facial emotion feature data, and voice emotion feature data; The Likert scale was used to encode the personalized psychological assessment questionnaire data and quantify them into questionnaire score values; Performing denoising and feature extraction preprocessing on the facial emotion feature data and the speech emotion feature data to generate speech emotion features and facial emotion features; Through feature-level fusion, the quantified questionnaire score values, voice emotion features, and facial emotion features are spliced ​​to form a comprehensive multimodal feature vector; The comprehensive multimodal feature vectors are combined with emotion recognition data to form a comprehensive personal psychological dataset containing self-report, voice emotion and facial emotion.

4. The method for screening and assessing mental health conditions as claimed in claim 3, characterized in that: The specific steps of generating a personalized mental health feedback report and improvement suggestions are as follows: Based on the predicted probability P(y|x) of the multimodal emotion-weighted random forest model, the expected probability of each psychological and physiological state y is calculated to construct a comprehensive health score , the expression is: ； Among them, I h (y) is the overall impact index of psychological and physiological state y on mental health; According to the feature importance and sentiment influence factor E of the random forest model i , identify the key psychophysiological indicators that have the greatest impact on current mental health status; Provide a set of personalized improvement strategies based on the identified key psychophysiological indicators.

5. A system for screening and evaluating mental health conditions, used to implement the method for screening and evaluating mental health conditions as claimed in any one of claims 1 to 4, characterized in that: Including questionnaire customization module, emotion recognition module, in-depth analysis module, mental health status assessment module and personalized suggestion report generation module; The questionnaire customization module is used to customize a personalized psychological assessment questionnaire through machine learning based on the user's initial information; The emotion recognition module is used to collect user voice and facial video for emotion recognition while the user is completing the questionnaire, and to construct comprehensive personal psychological data through multimodal data fusion; The deep analysis module is used to build an emotional cognitive neural network model based on a deep learning architecture combined with a convolutional neural network to deeply analyze personal psychological data and form analysis results of personal psychological and physiological indicators; The mental health status assessment module is used to combine the assessment feedback of the personalized psychological assessment questionnaire and the analysis results of personal psychological and physiological indicators to conduct a comprehensive assessment through a random forest model; The module for generating personalized suggestion reports is used to generate personalized mental health status feedback reports and improvement suggestions.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for general screening and assessment of mental health conditions described in any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for general screening and assessment of mental health conditions described in any one of claims 1 to 4 are implemented.

Images