Psychological health analysis method and system for psychological consultation

Abstract

The invention provides a psychological health analysis method and system for psychological counseling, and the method comprises the steps: obtaining original behavior data in a counseling process, extracting multi-modal features according to a fixed time window, and splicing the multi-modal features into a time sequence multi-modal feature set; constructing a cross-modal emotion consistency scoring model according to the expression features, the emotion features and the text semantic features, calculating a semantic-emotion consistency score and generating a global expression deviation factor; inputting the time sequence multi-modal feature set in combination with the semantic-emotion consistency score into a time sequence modeling structure with a modulation attention mechanism to calculate a psychological fluctuation intensity score and a psychological fluctuation index of the current time slice; and combining the global expression deviation factor, the psychological fluctuation intensity score of the current time slice and the psychological fluctuation index to generate a psychological risk score, and matching a corresponding personalized intervention suggestion based on the psychological risk score. The intelligent level of psychological risk identification and intervention decision making is comprehensively improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of psychological counseling, and particularly relates to a mental health analysis method and system for psychological counseling. BACKGROUND

[0002] With the continuous intensification of education pressure and the constant acceleration of social rhythm, the psychological burden faced by students in the growing process is increasingly complex and long-term, and mental health problems have become a key challenge that cannot be ignored in campus management and education system. Traditional psychological assessment methods mainly rely on artificial interviews, paper or electronic questionnaire scales and counselor experience judgment, although they have basic scientificity, but their evaluation dimension is single, the response cycle is lagging, and it is difficult to cover the dynamic change process of individuals, leading to the identification of psychological problems often lags behind the actual symptoms. In the campus environment, limited by the insufficient supply of psychological counseling resources, the limited expression ability of students, the implicitness of problems, and other factors, many individuals with psychological risks have difficulty in timely exposing problems, and the effectiveness of traditional methods in expression identification, process tracking and intervention matching is significantly restricted. In recent years, artificial intelligence and multi-modal perception technology have been gradually applied to mental health screening, and research directions have focused on identifying psychological states through voice tone, text emotion or facial expression modeling, but most of them only process a certain modality signal, lack of unified coding ability for multi-source behavior data, and cannot truly reflect the interactive structure of students' multi-modal expression. At the same time, existing systems are usually based on static analysis methods, which cannot reflect the time sequence evolution characteristics of students' psychological state, and cannot truly connect abnormal identification and intervention generation, resulting in that the technology is difficult to form a closed-loop system in the counseling scene. Especially in the face of the behavior characteristics of students' general expression suppression, emotional masking, and avoidance of communication, traditional models are difficult to perceive the potential risks of "inconsistency between language and non-language expression", further limiting the evaluation accuracy and intervention effectiveness.

[0003] Therefore, it is urgent to build an intelligent psychological analysis system for the campus psychological counseling scene, which has the ability of multi-modal behavior fusion, expression consistency modeling, dynamic state tracking and personalized intervention suggestion, in order to improve the response ability, perception depth and intervention efficiency of psychological services. SUMMARY

[0004] The purpose of the present application is to provide a mental health analysis method and system for psychological counseling to solve the above problems.

[0005] In order to achieve the above purpose, in the first aspect of the present application, a mental health analysis method for psychological counseling is provided, the method comprising the following steps:

[0006] The raw behavioral data during the consultation process is obtained, and multimodal features are extracted according to fixed time windows and concatenated into a time series multimodal feature set. Each time series multimodal feature represents a unified expression vector of the corresponding time segment, including facial expression features, emotional features, and textual semantic features.

[0007] Based on the facial expression features, emotional features, and text semantic features, a cross-modal emotion consistency scoring model is constructed to calculate the semantic-emotion consistency score and generate a global expression deviation factor.

[0008] The time series multimodal feature set is combined with the semantic-emotion consistency score input to the time series modeling structure with a modulated attention mechanism to calculate the psychological fluctuation intensity score and psychological volatility index of the current time segment;

[0009] A psychological risk score is generated by combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index. Based on the psychological risk score, corresponding personalized intervention suggestions are matched.

[0010] Furthermore, the raw behavioral data includes speech signals, image sequences, and text content.

[0011] Furthermore, the steps of acquiring raw behavioral data during the consultation process, extracting multimodal features according to a fixed time window, and concatenating them into a time-series multimodal feature set include:

[0012] Collect the audio signals, image sequences, and text content;

[0013] The speech signal, image sequence, and text content are preprocessed according to fixed time segments to obtain facial expression features, emotion features, and text semantic features.

[0014] The facial expression features, emotional features, and text semantic features are uniformly concatenated to obtain a unified feature vector that fully expresses the student's psychological state in the current time segment.

[0015] Furthermore, the preprocessing includes:

[0016] The image sequence selects all valid face image frames within the current time segment. An image feature extraction network containing five convolutional layers and two downsampling layers is used to process each image frame and extract a set of emotion-related features. Then, the feature vectors of all frames within the time segment are averaged over time to obtain the facial expression features under the facial expression modality.

[0017] The speech signal is divided into several overlapping frames within the current time segment. After extracting the Mel frequency cepstral coefficients, it is input into a temporal modeling network consisting of two convolutional layers and a gated recurrent unit, and the emotional features of the time segment are output.

[0018] The text data is first transcribed by a speech recognition system to extract all sentences within the current time segment. The transcribed data is then input into a Chinese semantic representation network consisting of twelve attention modules. The sentence embedding representation of the intermediate layer is extracted to form the text semantic features.

[0019] Furthermore, the step of constructing a cross-modal emotion consistency scoring model based on the facial expression features, emotion features, and text semantic features, calculating the semantic-emotion consistency score, and generating a global expression deviation factor includes:

[0020] Language emotion scores are extracted from the semantic features of the text through an emotion prediction network, with an output range of [0,1], where 0 represents extreme negative emotion and 1 represents extreme positive emotion;

[0021] The facial expression features and the emotional features are concatenated to form a non-verbal joint emotional feature. The non-verbal joint emotional feature is input into a convolutional and gated recurrent neural network to output a non-verbal emotional score. The output range is [0,1], where 0 represents an extremely negative emotion and 1 represents an extremely positive emotion.

[0022] By combining the verbal and nonverbal emotion scores, a cross-modal emotion consistency scoring model is constructed. The semantic-emotion consistency score is output through an exponential function, and a global expression deviation factor is calculated by calculating all semantic-emotion consistency scores. The closer the global expression deviation factor is to 1, the more severe the expression deviation of the student.

[0023] Furthermore, the emotion prediction network consists of three fully connected layers. The intermediate layers of the emotion prediction network are set as input layers 768 to 128, intermediate layers 128 to 32, and output layers 32 to 1. Each layer uses a linear transformation plus ReLU activation, and the last layer uses the Sigmoid function to normalize the output.

[0024] Furthermore, the step of calculating the psychological fluctuation intensity score and psychological volatility index of the current time segment by combining the time series multimodal feature set with the semantic-emotion consistency score input into the temporal modeling structure with a modulated attention mechanism includes:

[0025] The time series multimodal feature set of each time segment is fed into a set of linear transformation networks for dimensionality compression, so that it is mapped to a state vector with a dimension of 128; then the state vector is input into a bidirectional gated recurrent neural network to obtain forward state and backward state, which respectively model the accumulation and inversion features of psychological trend.

[0026] Based on the forward and reverse states, and combined with the nonlinear modulation weight of the semantic-emotion consistency score, the psychological fluctuation intensity score of the current time segment is calculated, with a numerical range of [-1, 1].

[0027] The psychological volatility index is calculated by performing second-order difference on the psychological volatility intensity score, which represents the degree of emotional stability of the user throughout the consultation process.

[0028] Furthermore, each direction of the bidirectional gated recurrent neural network contains two GRU units.

[0029] Furthermore, the step of generating a psychological risk score by combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index, and matching corresponding personalized intervention recommendations based on the psychological risk score includes:

[0030] By combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index, the psychological risk score of the current student is obtained by weighted summation.

[0031] Based on the aforementioned psychological risk score, risk levels are classified into low risk, medium risk, and high risk.

[0032] Based on the results of the risk level classification, a preset intervention suggestion template is matched.

[0033] A second aspect of the invention provides a mental health analysis system for psychological counseling, the system comprising:

[0034] The data acquisition unit is used to acquire raw behavioral data during the consultation process, extract multimodal features according to fixed time windows and concatenate them into a time series multimodal feature set; wherein, each time series multimodal feature represents a unified expression vector of the corresponding time segment, including facial expression features, emotion features and text semantic features;

[0035] The data analysis unit is used to construct a cross-modal emotion consistency scoring model based on the facial expression features, emotion features, and text semantic features, calculate the semantic-emotion consistency score, and generate a global expression deviation factor.

[0036] The psychological modality analysis unit is used to calculate the psychological fluctuation intensity score and psychological volatility index of the current time segment by combining the time series multimodal feature set with the semantic-emotion consistency score input into the time series modeling structure with a modulated attention mechanism;

[0037] The psychological report export unit is used to generate a psychological risk score by combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index, and to match corresponding personalized intervention suggestions based on the psychological risk score.

[0038] The beneficial technical effects of the present invention are at least as follows:

[0039] This invention proposes a mental health analysis method and system for student psychological counseling scenarios. It constructs a closed-loop structure around four technical links: multimodal data fusion, expression consistency modeling, dynamic evolution modeling of psychological states, and intervention generation linkage, comprehensively improving the intelligence level of psychological risk identification and intervention decision-making. The system first collects multimodal natural behavioral data of students during counseling through audio, video, and speech transcription channels, constructing a unified time-aligned expression vector sequence to ensure that various modalities are fused and expressed on the same analytical scale. Based on this, a cross-modal semantic-emotion consistency modeling module is constructed, quantifying the degree of psychological repression in expressive behavior through the degree of emotional deviation. In the subsequent dynamic modeling module of psychological states, consistency modulation terms and mutation sensitivity factors are introduced to achieve highly sensitive response and modeling of emotional inflection point behaviors, thereby constructing a psychological volatility index that reflects the true psychological trend of students. Finally, the system designs a risk scoring mechanism that integrates expression consistency, emotional intensity, and fluctuation speed to form a quantitative psychological risk level. Based on the score composition, it generates targeted personalized intervention suggestions, achieving a closed-loop output from identification to intervention. The overall solution takes into full consideration the implicit, variable and individual differences in the way students express their psychological problems. Through the synergistic design of expression modulation mechanism and risk-driven generation strategy, it enhances the system's ability to identify various types of risk individuals, such as those with high repression and low expression, and those with high volatility and irritability. It has good promotion and application value and feasibility for implementation. Attached Figure Description

[0040] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0041] Figure 1 This is a flowchart of the mental health analysis method for psychological counseling according to the present invention. Detailed Implementation

[0042] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0043] like Figure 1 As shown in the embodiment of the present invention, a mental health analysis method for psychological counseling is provided, the method comprising:

[0044] S1. Obtain the original behavioral data during the consultation process, extract multimodal features according to a fixed time window and concatenate them into a time series multimodal feature set; where each time series multimodal feature represents a unified expression vector of the corresponding time segment, including facial expression features, emotional features and text semantic features.

[0045] Specifically, this step aims to construct a structured representation of student behavioral data during psychological counseling interactions, providing a multi-source, synchronous, and computable input foundation for subsequent psychological state modeling. Targeting three common behavioral manifestations in psychological counseling scenarios—verbal expression, tone of voice, and facial expressions—this step utilizes audio, image, and transcribed text data, employing time segmentation, modality alignment, and embedding representation processing to generate time-series feature vectors with a unified structure. This processing fully considers the temporal synchronization and scale differences between modalities, employing a unified window slicing and standardized mapping method, enabling data from different modalities to be uniformly represented within the same computational structure, demonstrating good engineering feasibility.

[0046] Furthermore, this step serves as the data entry point for the entire system, with its input consisting of raw behavioral data from the consultation interaction process, specifically including the following three categories:

[0047] (1) Voice signal: The voice signal was collected through a digital microphone installed in the consultation equipment terminal. The sampling frequency was set to 16,000 times per second, and the signal was in mono uncompressed format. This data is used to reflect the characteristics of the student's intonation, speaking speed and rhythm in speech.

[0048] (2) Image sequence: Video image streams are captured at a rate of 15 frames per second using a high-definition camera positioned directly in front of the device. During the acquisition process, a face detection algorithm is used to filter out valid facial regions to ensure that the extracted facial expressions have clarity and temporal continuity.

[0049] (3) Text content: The speech signal is transcribed in real time by the speech recognition module to obtain Chinese short sentences with timestamps. The recognition system can use a continuous speech recognition network based on a self-attention mechanism to ensure the accuracy of language recognition and the rationality of sentence segmentation.

[0050] All three types of data are accompanied by a system timestamp and are time-aligned under the same clock system to ensure that the data expressed in different modalities fall within the same time segment.

[0051] Furthermore, the raw data is first divided into fixed time segments, with each segment recommended to last two seconds. Within each time segment, feature vectors representing the current psychological state are extracted from the three modalities.

[0052] Specifically as follows:

[0053] All valid face image frames within the current time segment are selected from the image data. An image feature extraction network containing five convolutional layers and two downsampling layers is used to process each frame and extract a set of emotion-related features. Subsequently, the feature vectors of all frames within the time segment are averaged over time to obtain the facial expression features under the facial expression modality, represented by symbols. express.

[0054] The speech signal is divided into several overlapping frames within the current time segment. After extracting the Mel-frequency cepstral coefficients, the signals are input into a temporal modeling network consisting of two convolutional layers and a gated recurrent unit. The network outputs the emotional features of that time segment, represented by symbols. express.

[0055] The text data is first transcribed using a speech recognition system to extract all sentences within the current time segment. These sentences are then input into a Chinese semantic representation network consisting of twelve attention modules. The sentence embeddings from the intermediate layers are extracted as text semantic features, denoted as .

[0056] Furthermore, after extracting features from the three modalities, the above features are concatenated to obtain a feature vector that fully expresses the student's psychological state in the current time segment:

[0057]

[0058] in:

[0059] Facial expression features, representing the feature vectors of facial expression modalities within the current time segment, are derived from image frames after passing through a convolutional feature extraction network and are averaged over time, with a dimension of 512.

[0060] Emotional features, representing feature vectors of speech signal modes, are derived from the output of audio signals after spectrum extraction and temporal network processing, and have a dimension of 128.

[0061] Text semantic features, which are feature vectors representing the semantic modalities of the text, are derived from the embeddings extracted after the speech transcription results are input into the language representation network, and have a dimension of 768.

[0062] z t The concatenated unified feature vector has a total dimension of 1408, where ∥ represents a vector-level concatenation operation;

[0063] All modal feature vectors have been normalized to zero mean and unit variance to prevent scale differences from affecting subsequent modeling.

[0064] Each vector z t Each time segment is accompanied by a timestamp 't' to maintain consistency in the time series.

[0065] For example, in a real ten-minute psychological counseling session, the entire interaction can be divided into three hundred time segments. Each time segment can be used to extract a complete multimodal feature vector, forming a time series feature set.

[0066] The output of this step is a time segment feature sequence. Where T represents the total number of time segments, z t This represents the unified psychological expression feature vector extracted from the t-th segment. This sequence will serve as the direct input for the subsequent expression consistency analysis module and the psychological state dynamic modeling module.

[0067] S2. Construct a cross-modal emotion consistency scoring model based on the facial expression features, emotion features, and text semantic features, calculate the semantic-emotion consistency score, and generate a global expression deviation factor.

[0068] Specifically, this step addresses the common problem of "inconsistency between verbal expression and emotional state" among students in psychological counseling scenarios. It proposes a cross-modal consistency modeling mechanism that integrates semantic and non-verbal modalities to identify potential emotional suppression, cognitive contradictions, or expressive concealment during counseling. Unlike traditional emotion recognition methods that only determine the user's emotional category, this step emphasizes a "multimodal expression consistency" approach, exploring the contradictory relationship between verbal content and non-verbal signals such as facial expressions and tone of voice. This emotional expression deviation is particularly common in student psychological problems and is an important clue for assessing their psychological stress and depressive tendencies. Therefore, this step introduces an expression deviation factor to provide psychological risk modulation signals for subsequent dynamic state modeling, forming a crucial intermediate link in the entire mental health assessment system.

[0069] Furthermore, the input for this step is the time-series multimodal feature set output from step one. Each z t This represents the unified representation vector for the t-th time segment, containing three types of modal subvectors:

[0070] The facial expression features of the facial image modality, with a dimension of 512, are obtained by frame averaging after image sequence processing through a convolutional network;

[0071] The emotional features of the speech modality, with a dimension of 128, are derived from the output of the Mel spectrogram input gated recurrent network.

[0072] The textual semantic features of the text modality, with a dimension of 768, are derived from the [CLS] embedding extracted from the speech transcription result input language representation network.

[0073] These features have been time-aligned and normalized in the previous stage and can be directly used for intermodal comparison and modeling.

[0074] Furthermore, the key step in this process is to construct a cross-modal emotion consistency scoring mechanism to measure the degree of consistency between students' verbal semantic emotion tendencies and their non-verbal emotional states (voice and facial expressions) within each time segment. To enhance the mechanism's sensitivity to the "emotional masking" phenomenon unique to psychological counseling scenarios, this step introduces a deviation penalty term into the traditional consistency measurement function and adds a semantic modality credibility adjustment mechanism to the consistency modeling structure.

[0075] First, from the semantic features of the text The algorithm extracts language emotion scores (s) through an emotion prediction network consisting of three fully connected layers. t Its output range is [0,1], where 0 represents extreme negative emotion and 1 represents extreme positive emotion. The intermediate layers of this network are set as follows: input layer 768→128, intermediate layer 128→32, output layer 32→1. Each layer uses linear transformation plus ReLU activation, and the last layer uses the Sigmoid function to normalize the output.

[0076] Then, facial features With emotional characteristics By splicing them together, non-verbal joint emotional features are formed. t Its dimension is 640. This feature is input into another two-branch convolutional and gated recurrent neural network, which outputs a non-verbal emotion score e. t ∈[0,1], the interpretation method is the same as s t same.

[0077] Furthermore, considering that students may exhibit defensive mechanisms or avoidant expressions in their language delivery, this invention designs the following consistency scoring function to enhance the model's ability to respond to such potential risks:

[0078] ct =exp(-α|s t -e t |·(1+β·(1-s t ) 2 (2)

[0079] Where: c t S is the semantic-sentiment consistency score for the t-th time segment; t To score the semantic sentiment of the text; e t The non-verbal modality is the emotion score; α is the basic consistency sensitivity factor, which can be set to 2; β is the negative semantic amplification coefficient, which can be set to 3.

[0080] The innovation of this function lies in: not only using |s t -e t |Measuring emotional disagreement, still expressed as negative emotions (s) t When the value approaches 0, the impact of consistency bias is further amplified, thus rapidly lowering c when students use negative language accompanied by non-verbal conflict. t This enhances the sensitivity of deviation detection.

[0081] In order to use the deviation of the entire consultation process as a quantitative indicator in subsequent modeling, the deviation factor C is defined as follows:

[0082]

[0083] In this formula, C represents the overall expression deviation factor, with a value range of [0,1], reflecting the overall degree of conflict between verbal and nonverbal emotions in an individual throughout the entire counseling session. The closer C is to 1, the more serious the expression deviation of the student, indicating a potential risk of emotional repression, self-contradiction, or active concealment.

[0084] In addition, to improve the consistency score c of the output t To assess the credibility, this step also marks time segments with low text transcription quality or missing language content as "low expression confidence" and records the mask vector during output for subsequent steps to adjust weights in dynamic modeling.

[0085] S3. Calculate the psychological fluctuation intensity score and psychological fluctuation index of the current time segment by combining the time series multimodal feature set with the semantic-emotion consistency score input into the time series modeling structure with the modulation attention mechanism.

[0086] Specifically, this step aims to construct an interpretable and modulated temporal psychological state modeling mechanism to address the dynamic changes in students' psychological states over time during counseling. This mechanism outputs the intensity of emotional fluctuations within each time segment and calculates the Psychological Volatility Index (PVI) for the student throughout the interaction process. This step employs a temporal modeling structure with an attention modulation mechanism, consisting of three parts: an input projection layer, a bidirectional gated loop structure, and a psychological state perception mapping layer. This structure not only captures long-term psychological trends but also achieves focused reinforcement of locally deviating segments by modulating channel fusion expression consistency scores.

[0087] Furthermore, firstly, the multimodal input z of each time segment is... t The vector is fed into a set of linear transformation networks for dimensionality compression, mapping it to the state vector h. t The dimension is 128. This state vector is then input into a bidirectional gated recurrent neural network (containing two GRU units in each direction) to obtain the forward state. Reverse state The accumulation and inversion characteristics of psychological trends are modeled separately.

[0088] Furthermore, when integrating the state vectors, this step introduces a semantic-emotion consistency score c. t The nonlinear modulation weights were used, and an "expression deviation sensitivity adjustment mechanism" suitable for psychodynamic modeling scenarios was designed, which was incorporated into the final state output in the model in the following form:

[0089]

[0090] Where: r t w represents the intensity score of psychological fluctuation in the t-th time segment, with a numerical range of [-1, 1]; r This is the state-weight vector, with a dimension of 128, used to integrate forward and reverse states; c t The consistency score is derived from step two; Δc t =c t -c t-1 The rate of change of consistency score is used to detect rapid abrupt changes in mood harmony; γ1 and γ2 are two adjustable hyperparameters that control consistency regulation and abrupt change sensitivity, respectively, with recommended values ​​of 1.5 and 0.8; the tanh(·) function restricts the output to [-1,1] to provide upper and lower bound interpretation.

[0091] This formula innovatively incorporates a term expressing the degree of deviation (1-c) in its structure. t ) 2 With deviation from speed term This allows the model to not only continuously monitor students with "long-term inconsistent expression" but also to dynamically respond to "expression abrupt change points." This design is applicable to "emotional inflection point behaviors" commonly seen in real-world psychological counseling scenarios, such as a student suddenly experiencing an emotional outburst from a calm state, characterized by intense speech and trembling tone.

[0092] Furthermore, after calculating the state scores for the entire sequence, to characterize the stability of an individual's psychological state throughout the entire process, the psychological volatility index (PVI) is defined as follows:

[0093]

[0094] This expression represents the absolute average of the second differences in a psychological state rating sequence, reflecting the "acceleration" level of psychological fluctuations and effectively capturing dramatic ups and downs rather than stable, gradual changes. Compared to the first difference (e.g., |r... t -r t-1 This design is more sensitive to capturing high-frequency oscillations, which aligns with the emotional fluctuation patterns of psychologically unstable individuals.

[0095] S4. Combine the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index to generate a psychological risk score, and match corresponding personalized intervention suggestions based on the psychological risk score.

[0096] Specifically, the core task of this step is to complete the psychological risk level assessment and intervention suggestion generation. Its functional positioning is the final decision output layer in the structure of this invention. It relies on the psychological state score results and psychological volatility index constructed in the previous stage, and based on these variables, it completes the quantitative classification of students' psychological risk level and generates targeted personalized intervention suggestions according to the risk level.

[0097] This invention first constructs a risk scoring index R, integrating the three input variables mentioned above. This score is used to reflect the overall psychological risk level of students during the current psychological counseling process. Considering r... t For time-series scoring, PVI represents the rate of change in emotion, and C represents the degree of deviation in expression. To prevent inconsistent variable dimensions from affecting the scoring bias, this step uses a normalized weighted scoring structure for fusion calculation:

[0098]

[0099] Where: R represents the student's psychological risk score; The overall intensity of the psychological state is represented by ω1, ω2, and ω3, which are the scoring weights. The default values ​​are 0.4, 0.4, and 0.2, and they can be fine-tuned based on historical data.

[0100] This scoring function introduces C 2 This item enhances the influence of expressive repression-type psychological risk in the assessment structure, ensuring that not only those with high volatility and high emotional intensity are judged as high risk, but also those potentially high-risk individuals with "low outward expression and high repression" can be identified, which is consistent with the expressive inhibition behavior characteristics commonly seen in student psychological counseling.

[0101] Next, risk levels are determined based on the risk score R. The classification criteria are three-tiered:

[0102] R<0.25: Low risk; the system is advised to automatically provide routine mental health advice.

[0103] 0.25≤R<0.45: Medium risk, a one-time follow-up visit or a short questionnaire reassessment is recommended;

[0104] R≥0.45: High risk. It is recommended to intervene in the professional consultation process and arrange psychological interviews or special assessments.

[0105] Finally, based on the risk level and the source of the constituent factors, a pre-defined intervention recommendation template is matched. The generation of intervention recommendations follows a two-factor mapping logic of "risk level + dominant factor". For example:

[0106] If the risk level is high risk, and C 2 If the item accounts for the largest proportion of the score (i.e., the main reason is suppressed expression), it is recommended to use the "guided expression intervention template", including open-ended dialogue, emotion card description exercises, etc.

[0107] If the risk level is medium and PVI is dominant, it is recommended to use the "emotion regulation schedule template", such as completing three deep breathing relaxation exercises within a week and checking in.

[0108] If the risk level is low, it is recommended that users read the psychological adjustment tips automatically pushed by the system and keep an emotional diary regularly.

[0109] The text of the intervention recommendation is automatically generated through a variable-filling template. The content is as follows: "This assessment shows that your mood changes are stable, but there are some suppressed characteristics in your expression. It is recommended to try emotion expression training activities, such as writing down a sentence of your feelings for the day each night to help gradually establish a mechanism for expressing inner feelings."

[0110] This invention also provides a mental health analysis system for psychological counseling, the system comprising:

[0111] The data acquisition unit is used to acquire raw behavioral data during the consultation process, extract multimodal features according to fixed time windows and concatenate them into a time series multimodal feature set; wherein, each time series multimodal feature represents a unified expression vector of the corresponding time segment, including facial expression features, emotion features and text semantic features;

[0112] The data analysis unit is used to construct a cross-modal emotion consistency scoring model based on the facial expression features, emotion features, and text semantic features, calculate the semantic-emotion consistency score, and generate a global expression deviation factor.

[0113] The psychological modality analysis unit is used to calculate the psychological fluctuation intensity score and psychological volatility index of the current time segment by combining the time series multimodal feature set with the semantic-emotion consistency score input into the time series modeling structure with a modulated attention mechanism;

[0114] The psychological report export unit is used to generate a psychological risk score by combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index, and to match corresponding personalized intervention suggestions based on the psychological risk score.

[0115] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0116] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or units may be electrical, mechanical, or other forms.

[0117] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0118] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A psychological health analysis method for use in psychological counseling, characterized in that, The method includes the following steps: The raw behavioral data during the consultation process is obtained, and multimodal features are extracted according to fixed time windows and concatenated into a time series multimodal feature set. Each time series multimodal feature represents a unified expression vector of the corresponding time segment, including facial expression features, emotional features, and textual semantic features. Based on the facial expression features, emotional features, and text semantic features, a cross-modal emotion consistency scoring model is constructed to calculate the semantic-emotion consistency score and generate a global expression deviation factor. The time series multimodal feature set is combined with the semantic-emotion consistency score input to the time series modeling structure with a modulated attention mechanism to calculate the psychological fluctuation intensity score and psychological volatility index of the current time segment; A psychological risk score is generated by combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index. Based on the psychological risk score, corresponding personalized intervention suggestions are matched.

2. The mental health analysis method for psychological counseling according to claim 1, characterized in that, The raw behavioral data includes voice signals, image sequences, and text content.

3. The mental health analysis method for psychological counseling according to claim 2, characterized in that, The steps of acquiring raw behavioral data during the consultation process, extracting multimodal features according to a fixed time window, and concatenating them into a time-series multimodal feature set include: Collect the audio signals, image sequences, and text content; The speech signal, image sequence, and text content are preprocessed according to fixed time segments to obtain facial expression features, emotion features, and text semantic features. The facial expression features, emotional features, and text semantic features are uniformly concatenated to obtain a unified feature vector that fully expresses the student's psychological state in the current time segment.

4. The mental health analysis method for psychological counseling according to claim 3, characterized in that, The preprocessing includes: The image sequence selects all valid face image frames within the current time segment. An image feature extraction network containing five convolutional layers and two downsampling layers is used to process each image frame and extract a set of emotion-related features. Then, the feature vectors of all frames within the time segment are averaged over time to obtain the facial expression features under the facial expression modality. The speech signal is divided into several overlapping frames within the current time segment. After extracting the Mel frequency cepstral coefficients, it is input into a temporal modeling network consisting of two convolutional layers and a gated recurrent unit, and the emotional features of the time segment are output. The text data is first transcribed by a speech recognition system to extract all sentences within the current time segment. The transcribed data is then input into a Chinese semantic representation network consisting of twelve attention modules. The sentence embedding representation of the intermediate layer is extracted to form the text semantic features.

5. The mental health analysis method for psychological counseling according to claim 1, characterized in that, The steps of constructing a cross-modal emotion consistency scoring model based on the facial expression features, emotional features, and text semantic features, calculating the semantic-emotion consistency score, and generating a global expression deviation factor include: Language emotion scores are extracted from the semantic features of the text through an emotion prediction network, with an output range of [0,1], where 0 represents extreme negative emotion and 1 represents extreme positive emotion; The facial expression features and the emotional features are concatenated to form a non-verbal joint emotional feature. The non-verbal joint emotional feature is input into a convolutional and gated recurrent neural network to output a non-verbal emotional score. The output range is [0,1], where 0 represents an extremely negative emotion and 1 represents an extremely positive emotion. By combining the verbal and nonverbal emotion scores, a cross-modal emotion consistency scoring model is constructed. The semantic-emotion consistency score is output through an exponential function, and a global expression deviation factor is calculated by calculating all semantic-emotion consistency scores. The closer the global expression deviation factor is to 1, the more severe the expression deviation of the student.

6. The mental health analysis method for psychological counseling according to claim 5, characterized in that, The emotion prediction network consists of three fully connected layers. The intermediate layers of the emotion prediction network are set as input layers 768 to 128, intermediate layers 128 to 32, and output layers 32 to 1. Each layer uses a linear transformation plus ReLU activation, and the last layer uses the Sigmoid function to normalize the output.

7. The mental health analysis method for psychological counseling according to claim 1, characterized in that, The step of calculating the psychological fluctuation intensity score and psychological volatility index of the current time segment by combining the time series multimodal feature set with the semantic-emotion consistency score input into the time series modeling structure with modulation attention mechanism includes: The time series multimodal feature set of each time segment is fed into a set of linear transformation networks for dimensionality compression, so that it is mapped to a state vector with a dimension of 128; then the state vector is input into a bidirectional gated recurrent neural network to obtain forward state and backward state, which respectively model the accumulation and inversion features of psychological trend. Based on the forward and reverse states, and combined with the nonlinear modulation weight of the semantic-emotion consistency score, the psychological fluctuation intensity score of the current time segment is calculated, with a numerical range of [-1, 1]. The psychological volatility index is calculated by performing second-order difference on the psychological volatility intensity score, which represents the degree of emotional stability of the user throughout the consultation process.

8. The mental health analysis method for psychological counseling according to claim 7, characterized in that, The bidirectional gated recurrent neural network contains two GRU units in each direction.

9. The mental health analysis method for psychological counseling according to claim 1, characterized in that, The step of generating a psychological risk score by combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index, and matching corresponding personalized intervention suggestions based on the psychological risk score includes: By combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index, the psychological risk score of the current student is obtained by weighted summation. Based on the aforementioned psychological risk score, risk levels are classified into low risk, medium risk, and high risk. Based on the results of the risk level classification, a preset intervention suggestion template is matched.

10. A mental health analysis system for psychological counseling, characterized in that, The system includes: The data acquisition unit is used to acquire raw behavioral data during the consultation process, extract multimodal features according to fixed time windows and concatenate them into a time series multimodal feature set; wherein, each time series multimodal feature represents a unified expression vector of the corresponding time segment, including facial expression features, emotion features and text semantic features; The data analysis unit is used to construct a cross-modal emotion consistency scoring model based on the facial expression features, emotion features, and text semantic features, calculate the semantic-emotion consistency score, and generate a global expression deviation factor. The psychological modality analysis unit is used to calculate the psychological fluctuation intensity score and psychological volatility index of the current time segment by combining the time series multimodal feature set with the semantic-emotion consistency score input into the time series modeling structure with a modulated attention mechanism; The psychological report export unit is used to generate a psychological risk score by combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index, and to match corresponding personalized intervention suggestions based on the psychological risk score.

Images