Mental health analysis method and system for psychological counseling

Abstract

The present application provides a mental health analysis method and system for psychological counseling, the method comprising: obtaining original behavior data in the counseling process, extracting multi-modal features according to a fixed time window and splicing into a time sequence multi-modal feature set; constructing a cross-modal emotion consistency scoring model according to the expression features, emotion features and text semantic features, calculating semantic-emotion consistency scores and generating a global expression deviation factor; inputting the time sequence multi-modal feature set combined with the semantic-emotion consistency scores into a time sequence modeling structure with a modulation attention mechanism to calculate the psychological fluctuation intensity score and the psychological fluctuation index of the current time segment; combining the global expression deviation factor, the psychological fluctuation intensity score and the psychological fluctuation index of the current time segment to generate a psychological risk score, and matching the corresponding personalized intervention suggestion based on the psychological risk score. The present application comprehensively improves the intelligent level of psychological risk identification and intervention decision-making.

Description

Technical Field

[0001] This invention belongs to the field of psychological counseling, and in particular relates to a psychological health analysis method and system for use in psychological counseling. Background Technology

[0002] With increasing educational pressure and a faster pace of life, students face increasingly complex and long-term psychological burdens during their growth, making mental health a critical challenge that cannot be ignored in school management and the education system. Traditional psychological assessment methods mainly rely on human interviews, paper or electronic questionnaires, and counselor experience. While these methods have basic scientific basis, their assessment dimensions are singular, their response cycles are lagging, and they fail to cover the dynamic changes in individuals, resulting in the identification of psychological problems often lagging behind the onset and aggravation of actual symptoms. In the school environment, due to insufficient supply of psychological counseling resources, students' limited expressive abilities, and the implicit nature of problems, many individuals at risk of psychological distress cannot expose their problems in a timely manner, significantly limiting the effectiveness of traditional methods in expression recognition, process tracking, and intervention matching. In recent years, artificial intelligence and multimodal perception technologies have been gradually applied to mental health screening. Research focuses on identifying psychological states through voice tone, text emotion, or facial expression modeling, but most of these methods only process signals of one modality and lack the ability to uniformly encode multi-source behavioral data, failing to truly reflect the interactive structure of students' multimodal expressions. Meanwhile, existing systems are typically based on static analysis methods, which fail to reflect the temporal evolution of students' psychological states and cannot truly connect anomaly identification with intervention, making it difficult for the technology to form an operational closed-loop system in counseling scenarios. In particular, when faced with students' common behavioral characteristics such as suppressed expression, emotional masking, and avoidance of communication, traditional models struggle to perceive the potential risk of inconsistency between verbal and nonverbal expressions, further limiting the accuracy of assessments and the effectiveness of interventions.

[0003] Therefore, there is an urgent need to build an intelligent psychological analysis system for campus psychological counseling scenarios, which has the capabilities of multimodal behavior fusion, expression consistency modeling, dynamic state tracking, and personalized intervention suggestions, in order to improve the responsiveness, perception depth, and intervention efficiency of psychological services. Summary of the Invention

[0004] The purpose of this invention is to propose a mental health analysis method and system for psychological counseling, thereby solving the above-mentioned problems.

[0005] To achieve the above objectives, a method for psychological health analysis in psychological counseling is provided in a first aspect of the present invention, the method comprising the following steps:

[0006] The raw behavioral data during the consultation process is obtained, and multimodal features are extracted according to fixed time windows and concatenated into a time series multimodal feature set. Each time series multimodal feature represents a unified expression vector of the corresponding time segment, including facial expression features, emotional features, and textual semantic features.

[0007] Based on the facial expression features, emotional features, and text semantic features, a cross-modal emotion consistency scoring model is constructed to calculate the semantic-emotion consistency score and generate a global expression deviation factor.

[0008] The time series multimodal feature set is combined with the semantic-emotion consistency score input to the time series modeling structure with a modulated attention mechanism to calculate the psychological fluctuation intensity score and psychological volatility index of the current time segment;

[0009] A psychological risk score is generated by combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index. Based on the psychological risk score, corresponding personalized intervention suggestions are matched.

[0010] Furthermore, the raw behavioral data includes speech signals, image sequences, and text content.

[0011] Furthermore, the steps of acquiring raw behavioral data during the consultation process, extracting multimodal features according to a fixed time window, and concatenating them into a time-series multimodal feature set include:

[0012] Collect the audio signals, image sequences, and text content;

[0013] The speech signal, image sequence, and text content are preprocessed according to fixed time segments to obtain facial expression features, emotion features, and text semantic features.

[0014] The facial expression features, emotional features, and text semantic features are uniformly concatenated to obtain a unified feature vector that fully expresses the student's psychological state in the current time segment.

[0015] Furthermore, the preprocessing includes:

[0016] The image sequence selects all valid face image frames within the current time segment. An image feature extraction network containing five convolutional layers and two downsampling layers is used to process each image frame and extract a set of emotion-related features. Then, the feature vectors of all frames within the time segment are averaged over time to obtain the facial expression features under the facial expression modality.

[0017] The speech signal is divided into several overlapping frames within the current time segment. After extracting the Mel frequency cepstral coefficients, it is input into a temporal modeling network consisting of two convolutional layers and a gated recurrent unit, and the emotional features of the time segment are output.

[0018] The text data is first transcribed by a speech recognition system to extract all sentences within the current time segment. The transcribed data is then input into a Chinese semantic representation network consisting of twelve attention modules. The sentence embedding representation of the intermediate layer is extracted to form the text semantic features.

[0019] Furthermore, the step of constructing a cross-modal emotion consistency scoring model based on the facial expression features, emotion features, and text semantic features, calculating the semantic-emotion consistency score, and generating a global expression deviation factor includes:

[0020] Language emotion scores are extracted from the semantic features of the text through an emotion prediction network, with an output range of [0,1], where 0 represents extreme negative emotion and 1 represents extreme positive emotion;

[0021] The facial expression features and the emotional features are concatenated to form a non-verbal joint emotional feature. The non-verbal joint emotional feature is input into a convolutional and gated recurrent neural network to output a non-verbal emotional score. The output range is [0,1], where 0 represents an extremely negative emotion and 1 represents an extremely positive emotion.

[0022] By combining the verbal and nonverbal emotion scores, a cross-modal emotion consistency scoring model is constructed. The semantic-emotion consistency score is output through an exponential function, and a global expression deviation factor is calculated by calculating all semantic-emotion consistency scores. The closer the global expression deviation factor is to 1, the more severe the expression deviation of the student.

[0023] Furthermore, the emotion prediction network consists of three fully connected layers. The intermediate layers of the emotion prediction network are set as input layers 768 to 128, intermediate layers 128 to 32, and output layers 32 to 1. Each layer uses a linear transformation plus ReLU activation, and the last layer uses the Sigmoid function to normalize the output.

[0024] Furthermore, the step of calculating the psychological fluctuation intensity score and psychological volatility index of the current time segment by combining the time series multimodal feature set with the semantic-emotion consistency score input into the temporal modeling structure with a modulated attention mechanism includes:

[0025] The time series multimodal feature set of each time segment is fed into a set of linear transformation networks for dimensionality compression, so that it is mapped to a state vector with a dimension of 128; then the state vector is input into a bidirectional gated recurrent neural network to obtain forward state and backward state, which respectively model the accumulation and inversion features of psychological trend.

[0026] Based on the forward and reverse states, and combined with the nonlinear modulation weight of the semantic-emotion consistency score, the psychological fluctuation intensity score of the current time segment is calculated, with a numerical range of [-1, 1].

[0027] The psychological volatility index is calculated by performing second-order difference on the psychological volatility intensity score, which represents the degree of emotional stability of the user throughout the consultation process.

[0028] Furthermore, each direction of the bidirectional gated recurrent neural network contains two GRU units.

[0029] Furthermore, the step of generating a psychological risk score by combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index, and matching corresponding personalized intervention recommendations based on the psychological risk score includes:

[0030] By combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index, the psychological risk score of the current student is obtained by weighted summation.

[0031] Based on the aforementioned psychological risk score, risk levels are classified into low risk, medium risk, and high risk.

[0032] Based on the results of the risk level classification, a preset intervention suggestion template is matched.

[0033] A second aspect of the invention provides a mental health analysis system for psychological counseling, the system comprising:

[0034] The data acquisition unit is used to acquire raw behavioral data during the consultation process, extract multimodal features according to fixed time windows and concatenate them into a time series multimodal feature set; wherein, each time series multimodal feature represents a unified expression vector of the corresponding time segment, including facial expression features, emotion features and text semantic features;

[0035] The data analysis unit is used to construct a cross-modal emotion consistency scoring model based on the facial expression features, emotion features, and text semantic features, calculate the semantic-emotion consistency score, and generate a global expression deviation factor.

[0036] The psychological modality analysis unit is used to calculate the psychological fluctuation intensity score and psychological volatility index of the current time segment by combining the time series multimodal feature set with the semantic-emotion consistency score input into the time series modeling structure with a modulated attention mechanism;

[0037] The psychological report export unit is used to generate a psychological risk score by combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index, and to match corresponding personalized intervention suggestions based on the psychological risk score.

[0038] The beneficial technical effects of the present invention are at least as follows:

[0039] This invention proposes a mental health analysis method and system for student psychological counseling scenarios. It constructs a closed-loop structure around four technical links: multimodal data fusion, expression consistency modeling, dynamic evolution modeling of psychological states, and intervention generation linkage, comprehensively improving the intelligence level of psychological risk identification and intervention decision-making. The system first collects multimodal natural behavioral data of students during counseling through audio, video, and speech transcription channels, constructing a unified time-aligned expression vector sequence to ensure that various modalities are fused and expressed on the same analytical scale. Based on this, a cross-modal semantic-emotion consistency modeling module is constructed, quantifying the degree of psychological repression in expressive behavior through the degree of emotional deviation. In the subsequent dynamic modeling module of psychological states, consistency modulation terms and mutation sensitivity factors are introduced to achieve highly sensitive response and modeling of emotional inflection point behaviors, thereby constructing a psychological volatility index that reflects the true psychological trend of students. Finally, the system designs a risk scoring mechanism that integrates expression consistency, emotional intensity, and fluctuation speed to form a quantitative psychological risk level. Based on the score composition, it generates targeted personalized intervention suggestions, achieving a closed-loop output from identification to intervention. The overall solution takes into full consideration the implicit, variable and individual differences in the way students express their psychological problems. Through the synergistic design of expression modulation mechanism and risk-driven generation strategy, it enhances the system's ability to identify various types of risk individuals, such as those with high repression and low expression, and those with high volatility and irritability. It has good promotion and application value and feasibility for implementation. Attached Figure Description

[0040] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0041] Figure 1 This is a flowchart of the mental health analysis method for psychological counseling according to the present invention. Detailed Implementation

[0042] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0043] like Figure 1 As shown in the embodiment of the present invention, a mental health analysis method for psychological counseling is provided, the method comprising:

[0044] S1. Obtain the original behavioral data during the consultation process, extract multimodal features according to a fixed time window and concatenate them into a time series multimodal feature set; where each time series multimodal feature represents a unified expression vector of the corresponding time segment, including facial expression features, emotional features and text semantic features.

[0045] Specifically, this step aims to construct a structured representation of student behavioral data during psychological counseling interactions, providing a multi-source, synchronous, and computable input foundation for subsequent psychological state modeling. Targeting three common behavioral manifestations in psychological counseling scenarios—verbal expression, tone of voice, and facial expressions—this step utilizes audio, image, and transcribed text data, employing time segmentation, modality alignment, and embedding representation processing to generate time-series feature vectors with a unified structure. This processing fully considers the temporal synchronization and scale differences between modalities, employing a unified window slicing and standardized mapping method, enabling data from different modalities to be uniformly represented within the same computational structure, demonstrating good engineering feasibility.

[0046] Furthermore, this step serves as the data entry point for the entire system, with its input consisting of raw behavioral data from the consultation interaction process, specifically including the following three categories:

[0047] (1) Voice signal: The voice signal was collected through a digital microphone installed in the consultation equipment terminal. The sampling frequency was set to 16,000 times per second, and the signal was in mono uncompressed format. This data is used to reflect the characteristics of the student's intonation, speaking speed and rhythm in speech.

[0048] (2) Image sequence: Video image streams are captured at a rate of 15 frames per second using a high-definition camera positioned directly in front of the device. During the acquisition process, a face detection algorithm is used to filter out valid facial regions to ensure that the extracted facial expressions have clarity and temporal continuity.

[0049] (3) Text content: The speech signal is transcribed in real time by the speech recognition module to obtain Chinese short sentences with timestamps. The recognition system can use a continuous speech recognition network based on a self-attention mechanism to ensure the accuracy of language recognition and the rationality of sentence segmentation.

[0050] All three types of data are accompanied by a system timestamp and are time-aligned under the same clock system to ensure that the data expressed in different modalities fall within the same time segment.

[0051] Furthermore, the raw data is first divided into fixed time segments, with each segment recommended to last two seconds. Within each time segment, feature vectors representing the current psychological state are extracted from the three modalities.

[0052] Specifically as follows:

[0053] All valid face image frames within the current time segment are selected from the image data. An image feature extraction network containing five convolutional layers and two downsampling layers is used to process each frame and extract a set of emotion-related features. Subsequently, the feature vectors of all frames within the time segment are averaged over time to obtain the facial expression features under the facial expression modality, represented by symbols. express.

[0054] The speech signal is divided into several overlapping frames within the current time segment. After extracting the Mel-frequency cepstral coefficients, the signals are input into a temporal modeling network consisting of two convolutional layers and a gated recurrent unit. The network outputs the emotional features of that time segment, represented by symbols. express.

[0055] The text data is first transcribed using a speech recognition system to extract all sentences within the current time segment. These sentences are then input into a Chinese semantic representation network consisting of twelve attention modules. The sentence embeddings from the intermediate layers are extracted as text semantic features, denoted as .

[0056] Furthermore, after extracting features from the three modalities, the above features are concatenated to obtain a feature vector that fully expresses the student's psychological state in the current time segment:

[0057]

[0058] in:

[0059] Facial expression features, representing the feature vectors of facial expression modalities within the current time segment, are derived from image frames after passing through a convolutional feature extraction network and are averaged over time, with a dimension of 512.

[0060] Emotional features, representing feature vectors of speech signal modes, are derived from the output of audio signals after spectrum extraction and temporal network processing, and have a dimension of 128.

[0061] Text semantic features, which are feature vectors representing the semantic modalities of the text, are derived from the embeddings extracted after the speech transcription results are input into the language representation network, and have a dimension of 768.

[0062] z t The concatenated unified feature vector has a total dimension of 1408, where ∥ represents a vector-level concatenation operation;

[0063] All modal feature vectors have been normalized to zero mean and unit variance to prevent scale differences from affecting subsequent modeling.

[0064] Each vector z t Each time segment is accompanied by a timestamp 't' to maintain consistency in the time series.

[0065] For example, in a real ten-minute psychological counseling session, the entire interaction can be divided into three hundred time segments. Each time segment can be used to extract a complete multimodal feature vector, forming a time series feature set.

[0066] The output of this step is a time segment feature sequence. Where T represents the total number of time segments, z t This represents the unified psychological expression feature vector extracted from the t-th segment. This sequence will serve as the direct input for the subsequent expression consistency analysis module and the psychological state dynamic modeling module.

[0067] S2. Construct a cross-modal emotion consistency scoring model based on the facial expression features, emotion features, and text semantic features, calculate the semantic-emotion consistency score, and generate a global expression deviation factor.

[0068] Specifically, this step addresses the common problem of "inconsistency between verbal expression and emotional state" among students in psychological counseling scenarios. It proposes a cross-modal consistency modeling mechanism that integrates semantic and non-verbal modalities to identify potential emotional suppression, cognitive contradictions, or expressive concealment during counseling. Unlike traditional emotion recognition methods that only determine the user's emotional category, this step emphasizes a "multimodal expression consistency" approach, exploring the contradictory relationship between verbal content and non-verbal signals such as facial expressions and tone of voice. This emotional expression deviation is particularly common in student psychological problems and is an important clue for assessing their psychological stress and depressive tendencies. Therefore, this step introduces an expression deviation factor to provide psychological risk modulation signals for subsequent dynamic state modeling, forming a crucial intermediate link in the entire mental health assessment system.

[0069] Furthermore, the input for this step is the time-series multimodal feature set output from step one. Each z t This represents the unified representation vector for the t-th time segment, containing three types of modal subvectors:

[0070] The facial expression features of the facial image modality, with a dimension of 512, are obtained by frame averaging after image sequence processing through a convolutional network;

[0071] The emotional features of the speech modality, with a dimension of 128, are derived from the output of the Mel spectrogram input gated recurrent network.

[0072] The textual semantic features of the text modality, with a dimension of 768, are derived from the [CLS] embedding extracted from the speech transcription result input language representation network.

[0073] These features have been time-aligned and normalized in the previous stage and can be directly used for intermodal comparison and modeling.

[0074] Furthermore, the key step in this process is to construct a cross-modal emotion consistency scoring mechanism to measure the degree of consistency between students' verbal semantic emotion tendencies and their non-verbal emotional states (voice and facial expressions) within each time segment. To enhance the mechanism's sensitivity to the "emotional masking" phenomenon unique to psychological counseling scenarios, this step introduces a deviation penalty term into the traditional consistency measurement function and adds a semantic modality credibility adjustment mechanism to the consistency modeling structure.

[0075] First, from the semantic features of the text The algorithm extracts language emotion scores (s) through an emotion prediction network consisting of three fully connected layers. t Its output range is [0,1], where 0 represents extreme negative emotion and 1 represents extreme positive emotion. The intermediate layers of this network are set as follows: input layer 768→128, intermediate layer 128→32, output layer 32→1. Each layer uses linear transformation plus ReLU activation, and the last layer uses the Sigmoid function to normalize the output.

[0076] Then, facial features With emotional characteristics By splicing them together, non-verbal joint emotional features are formed. t Its dimension is 640. This feature is input into another two-branch convolutional and gated recurrent neural network, which outputs a non-verbal emotion score e. t ∈[0,1], the interpretation method is the same as s t same.

[0077] Furthermore, considering that students may exhibit defensive mechanisms or avoidant expressions in their language delivery, this invention designs the following consistency scoring function to enhance the model's ability to respond to such potential risks:

[0078] ct =exp(-α|s t -e t |·(1+β·(1-s t ) 2 (2)

[0079] Where: c t S is the semantic-sentiment consistency score for the t-th time segment; t To score the semantic sentiment of the text; e t The non-verbal modality is the emotion score; α is the basic consistency sensitivity factor, which can be set to 2; β is the negative semantic amplification coefficient, which can be set to 3.

[0080] The innovation of this function lies in: not only using |s t -e t |Measuring emotional disagreement, still expressed as negative emotions (s) t When the value approaches 0, the impact of consistency bias is further amplified, thus rapidly lowering c when students use negative language accompanied by non-verbal conflict. t This enhances the sensitivity of deviation detection.

[0081] In order to use the deviation of the entire consultation process as a quantitative indicator in subsequent modeling, the deviation factor C is defined as follows:

[0082]

[0083] In this formula, C represents the overall expression deviation factor, with a value range of [0,1], reflecting the overall degree of conflict between verbal and nonverbal emotions in an individual throughout the entire counseling session. The closer C is to 1, the more serious the expression deviation of the student, indicating a potential risk of emotional repression, self-contradiction, or active concealment.

[0084] In addition, to improve the consistency score c of the output t To assess the credibility, this step also marks time segments with low text transcription quality or missing language content as "low expression confidence" and records the mask vector during output for subsequent steps to adjust weights in dynamic modeling.

[0085] S3. Calculate the psychological fluctuation intensity score and psychological fluctuation index of the current time segment by combining the time series multimodal feature set with the semantic-emotion consistency score input into the time series modeling structure with the modulation attention mechanism.

[0086] Specifically, this step aims to construct an interpretable and modulated temporal psychological state modeling mechanism to address the dynamic changes in students' psychological states over time during counseling. This mechanism outputs the intensity of emotional fluctuations within each time segment and calculates the Psychological Volatility Index (PVI) for the student throughout the interaction process. This step employs a temporal modeling structure with an attention modulation mechanism, consisting of three parts: an input projection layer, a bidirectional gated loop structure, and a psychological state perception mapping layer. This structure not only captures long-term psychological trends but also achieves focused reinforcement of locally deviating segments by modulating channel fusion expression consistency scores.

[0087] Furthermore, firstly, the multimodal input z of each time segment is... t The vector is fed into a set of linear transformation networks for dimensionality compression, mapping it to the state vector h. t The dimension is 128. This state vector is then input into a bidirectional gated recurrent neural network (containing two GRU units in each direction) to obtain the forward state. Reverse state The accumulation and inversion characteristics of psychological trends are modeled separately.

[0088] Furthermore, when integrating the state vectors, this step introduces a semantic-emotion consistency score c. t The nonlinear modulation weights were used, and an "expression deviation sensitivity adjustment mechanism" suitable for psychodynamic modeling scenarios was designed, which was incorporated into the final state output in the model in the following form:

[0089]

[0090] Where: r t w represents the intensity score of psychological fluctuation in the t-th time segment, with a numerical range of [-1, 1]; r This is the state-weight vector, with a dimension of 128, used to integrate forward and reverse states; c t The consistency score is derived from step two; Δc t =c t -c t-1 The rate of change of consistency score is used to detect rapid abrupt changes in mood harmony; γ1 and γ2 are two adjustable hyperparameters that control consistency regulation and abrupt change sensitivity, respectively, with recommended values ​​of 1.5 and 0.8; the tanh(·) function restricts the output to [-1,1] to provide upper and lower bound interpretation.

[0091] This formula innovatively incorporates a term expressing the degree of deviation (1-c) in its structure. t ) 2 With deviation from speed term This allows the model to not only continuously monitor students with "long-term inconsistent expression" but also to dynamically respond to "expression abrupt change points." This design is applicable to "emotional inflection point behaviors" commonly seen in real-world psychological counseling scenarios, such as a student suddenly experiencing an emotional outburst from a calm state, characterized by intense speech and trembling tone.

[0092] Furthermore, after calculating the state scores for the entire sequence, to characterize the stability of an individual's psychological state throughout the entire process, the psychological volatility index (PVI) is defined as follows:

[0093]

[0094] This expression represents the absolute average of the second differences in a psychological state rating sequence, reflecting the "acceleration" level of psychological fluctuations and effectively capturing dramatic ups and downs rather than stable, gradual changes. Compared to the first difference (e.g., |r... t -r t-1 This design is more sensitive to capturing high-frequency oscillations, which aligns with the emotional fluctuation patterns of psychologically unstable individuals.

[0095] S4. Combine the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index to generate a psychological risk score, and match corresponding personalized intervention suggestions based on the psychological risk score.

[0096] Specifically, the core task of this step is to complete the psychological risk level assessment and intervention suggestion generation. Its functional positioning is the final decision output layer in the structure of this invention. It relies on the psychological state score results and psychological volatility index constructed in the previous stage, and based on these variables, it completes the quantitative classification of students' psychological risk level and generates targeted personalized intervention suggestions according to the risk level.

[0097] This invention first constructs a risk scoring index R, integrating the three input variables mentioned above. This score is used to reflect the overall psychological risk level of students during the current psychological counseling process. Considering r... t For time-series scoring, PVI represents the rate of change in emotion, and C represents the degree of deviation in expression. To prevent inconsistent variable dimensions from affecting the scoring bias, this step uses a normalized weighted scoring structure for fusion calculation:

[0098]

[0099] Where: R represents the student's psychological risk score; The overall intensity of the psychological state is represented by ω1, ω2, and ω3, which are the scoring weights. The default values ​​are 0.4, 0.4, and 0.2, and they can be fine-tuned based on historical data.

[0100] This scoring function introduces C 2 This item enhances the influence of expressive repression-type psychological risk in the assessment structure, ensuring that not only those with high volatility and high emotional intensity are judged as high risk, but also those potentially high-risk individuals with "low outward expression and high repression" can be identified, which is consistent with the expressive inhibition behavior characteristics commonly seen in student psychological counseling.

[0101] Next, risk levels are determined based on the risk score R. The classification criteria are three-tiered:

[0102] R<0.25: Low risk; the system is advised to automatically provide routine mental health advice.

[0103] 0.25≤R<0.45: Medium risk, a one-time follow-up visit or a short questionnaire reassessment is recommended;

[0104] R≥0.45: High risk. It is recommended to intervene in the professional consultation process and arrange psychological interviews or special assessments.

[0105] Finally, based on the risk level and the source of the constituent factors, a pre-defined intervention recommendation template is matched. The generation of intervention recommendations follows a two-factor mapping logic of "risk level + dominant factor". For example:

[0106] If the risk level is high risk, and C 2 If the item accounts for the largest proportion of the score (i.e., the main reason is suppressed expression), it is recommended to use the "guided expression intervention template", including open-ended dialogue, emotion card description exercises, etc.

[0107] If the risk level is medium and PVI is dominant, it is recommended to use the "emotion regulation schedule template", such as completing three deep breathing relaxation exercises within a week and checking in.

[0108] If the risk level is low, it is recommended that users read the psychological adjustment tips automatically pushed by the system and keep an emotional diary regularly.

[0109] The text of the intervention recommendation is automatically generated through a variable-filling template. The content is as follows: "This assessment shows that your mood changes are stable, but there are some suppressed characteristics in your expression. It is recommended to try emotion expression training activities, such as writing down a sentence of your feelings for the day each night to help gradually establish a mechanism for expressing inner feelings."

[0110] This invention also provides a mental health analysis system for psychological counseling, the system comprising:

[0111] The data acquisition unit is used to acquire raw behavioral data during the consultation process, extract multimodal features according to fixed time windows and concatenate them into a time series multimodal feature set; wherein, each time series multimodal feature represents a unified expression vector of the corresponding time segment, including facial expression features, emotion features and text semantic features;

[0112] The data analysis unit is used to construct a cross-modal emotion consistency scoring model based on the facial expression features, emotion features, and text semantic features, calculate the semantic-emotion consistency score, and generate a global expression deviation factor.

[0113] The psychological modality analysis unit is used to calculate the psychological fluctuation intensity score and psychological volatility index of the current time segment by combining the time series multimodal feature set with the semantic-emotion consistency score input into the time series modeling structure with a modulated attention mechanism;

[0114] The psychological report export unit is used to generate a psychological risk score by combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index, and to match corresponding personalized intervention suggestions based on the psychological risk score.

[0115] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0116] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or units may be electrical, mechanical, or other forms.

[0117] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0118] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A psychological health analysis method for use in psychological counseling, characterized in that, The method includes the following steps: The raw behavioral data during the consultation process is obtained, and multimodal features are extracted according to fixed time windows and concatenated into a time series multimodal feature set. Each time series multimodal feature represents a unified expression vector of the corresponding time segment, including facial expression features, emotional features, and textual semantic features. Based on the facial expression features, emotional features, and text semantic features, a cross-modal emotion consistency scoring model is constructed to calculate the semantic-emotion consistency score and generate a global expression deviation factor. The time series multimodal feature set is combined with the semantic-emotion consistency score input to the time series modeling structure with a modulated attention mechanism to calculate the psychological fluctuation intensity score and psychological volatility index of the current time segment; A psychological risk score is generated by combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index. Based on the psychological risk score, corresponding personalized intervention suggestions are matched. The steps of constructing a cross-modal emotion consistency scoring model based on the facial expression features, emotional features, and text semantic features, calculating the semantic-emotion consistency score, and generating a global expression deviation factor include: The language sentiment score is extracted from the semantic features of the text using a sentiment prediction network, and its output range is as follows: , where 0 represents extreme negative emotion and 1 represents extreme positive emotion; The facial expression features and the emotion features are concatenated to form a nonverbal joint emotion feature. This nonverbal joint emotion feature is then input into a convolutional and gated recurrent neural network to output a nonverbal emotion score, the output range of which is [missing information]. , where 0 represents extreme negative emotion and 1 represents extreme positive emotion; By combining the verbal and nonverbal emotion scores, a cross-modal emotion consistency scoring model is constructed. The semantic-emotion consistency score is output through an exponential function, and a global expression deviation factor is calculated by calculating all semantic-emotion consistency scores. The closer the global expression deviation factor is to 1, the more severe the expression deviation of the current student.

2. The mental health analysis method for psychological counseling according to claim 1, characterized in that, The raw behavioral data includes voice signals, image sequences, and text content.

3. The mental health analysis method for psychological counseling according to claim 2, characterized in that, The steps of acquiring raw behavioral data during the consultation process, extracting multimodal features according to a fixed time window, and concatenating them into a time-series multimodal feature set include: Collect the audio signals, image sequences, and text content; The speech signal, image sequence, and text content are preprocessed according to fixed time segments to obtain facial expression features, emotion features, and text semantic features. The facial expression features, emotional features, and text semantic features are uniformly concatenated to obtain a unified feature vector that fully expresses the student's psychological state in the current time segment.

4. The mental health analysis method for psychological counseling according to claim 3, characterized in that, The preprocessing includes: The image sequence selects all valid face image frames within the current time segment. An image feature extraction network containing five convolutional layers and two downsampling layers is used to process each image frame and extract a set of emotion-related features. Then, the feature vectors of all frames within the time segment are averaged over time to obtain the facial expression features under the facial expression modality. The speech signal is divided into several overlapping frames within the current time segment. After extracting the Mel frequency cepstral coefficients, it is input into a temporal modeling network consisting of two convolutional layers and a gated recurrent unit, and the emotional features of the time segment are output. The text content is first transcribed by a speech recognition system to extract all sentences within the current time segment. It is then input into a Chinese semantic representation network consisting of twelve attention modules, and the sentence embedding representation of its intermediate layers is extracted as the text semantic features.

5. The mental health analysis method for psychological counseling according to claim 1, characterized in that, The emotion prediction network consists of three fully connected layers. The intermediate layers of the emotion prediction network are set as input layers 768 to 128, intermediate layers 128 to 32, and output layers 32 to 1. Each layer uses a linear transformation plus ReLU activation, and the last layer uses the Sigmoid function to normalize the output.

6. The mental health analysis method for psychological counseling according to claim 1, characterized in that, The step of calculating the psychological fluctuation intensity score and psychological volatility index of the current time segment by combining the time series multimodal feature set with the semantic-emotion consistency score input into the time series modeling structure with modulation attention mechanism includes: The time series multimodal feature set of each time segment is fed into a set of linear transformation networks for dimensionality compression, so that it is mapped to a state vector with a dimension of 128; then the state vector is input into a bidirectional gated recurrent neural network to obtain forward state and backward state, which respectively model the accumulation and inversion features of psychological trend. Based on the forward and reverse states, and combined with the nonlinear modulation weights of the semantic-emotion consistency score, the psychological fluctuation intensity score for the current time segment is calculated, with a numerical range of [missing value]. ; The psychological volatility index is calculated by performing second-order difference on the psychological volatility intensity score, which represents the degree of emotional stability of the user throughout the consultation process.

7. The mental health analysis method for psychological counseling according to claim 6, characterized in that, The bidirectional gated recurrent neural network contains two GRU units in each direction.

8. The mental health analysis method for psychological counseling according to claim 1, characterized in that, The step of generating a psychological risk score by combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index, and matching corresponding personalized intervention suggestions based on the psychological risk score includes: By combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index, the psychological risk score of the current student is obtained by weighted summation. Based on the aforementioned psychological risk score, risk levels are classified into low risk, medium risk, and high risk. Based on the results of the risk level classification, a preset intervention suggestion template is matched.

9. A system for performing the mental health analysis method for psychological counseling as described in claim 1, characterized in that, The system includes: The data acquisition unit is used to acquire raw behavioral data during the consultation process, extract multimodal features according to fixed time windows and concatenate them into a time series multimodal feature set; wherein, each time series multimodal feature represents a unified expression vector of the corresponding time segment, including facial expression features, emotion features and text semantic features; The data analysis unit is used to construct a cross-modal emotion consistency scoring model based on the facial expression features, emotion features, and text semantic features, calculate the semantic-emotion consistency score, and generate a global expression deviation factor. The psychological modality analysis unit is used to calculate the psychological fluctuation intensity score and psychological volatility index of the current time segment by combining the time series multimodal feature set with the semantic-emotion consistency score input into the time series modeling structure with a modulated attention mechanism; The psychological report export unit is used to generate a psychological risk score by combining the global expression deviation factor, the psychological fluctuation intensity score of the current time segment, and the psychological volatility index, and to match corresponding personalized intervention suggestions based on the psychological risk score.

Images