Personalized Student Psychological Counseling Methods and Systems Based on Grouped Hybrid Expert Models

Abstract

This invention discloses a personalized student psychological counseling method and system based on a grouped hybrid expert model, belonging to the field of natural language processing. The method first rewrites single-turn question-and-answer data into multi-turn dialogues and embeds counselor role features to form dialogue text data. Then, it extracts the problem context, role labels, and supporting strategies, training a role matching model and a strategy prediction model respectively. A grouped hybrid expert model is constructed by improving the LLaMA model structure, and a gating network is used to achieve routing learning of role and strategy features. In the inference stage, the counselor role and supporting strategy labels are determined through the two prediction models. An initial response is generated by the grouped hybrid expert model, and then supplemented by relevant experiences retrieved from a pre-built role experience database, ultimately forming a personalized counseling response returned to the student client. This method achieves an organic combination of counselor role matching, supporting strategy prediction, and emotional response generation.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing, and in particular relates to a personalized student psychological counseling method and system based on a grouped hybrid expert model. Background Technology

[0002] Mental health is a crucial component of overall human health, directly impacting an individual's well-being, learning efficiency, and social adaptability. Since the beginning of the 21st century, with increasing academic pressure, fierce competition for higher education, and a more complex social environment, mental health issues among students have become increasingly prominent. According to the World Health Organization's (WHO) 2022 World Mental Health Report, approximately one in eight people globally suffers from a mental disorder, with adolescents and students being among the high-risk groups. However, limited by high counseling costs and a shortage of professional mental health resources on campuses, student mental health services face significant challenges in terms of accessibility and coverage, urgently requiring the use of artificial intelligence (AI) technology to improve intervention efficiency and service accessibility.

[0003] Fortunately, the rapid development of Natural Language Processing (NLP) technology in recent years has provided a potential solution to this core problem. In particular, the rapid progress of Large Language Models (LLMs), such as GPT-4, Claude-3.5, and LLaMA3, has achieved remarkable results in dialogue generation and emotional support, and has initially demonstrated the ability to provide students with psychological comfort and advice.

[0004] However, these models generally adopt a neutral, assistant-like response style, lacking a distinctive language style and deep empathy, making it difficult to meet users' needs for emotional support. To address this deficiency, some studies have attempted to integrate psychological counseling theories into dialogue systems. For example, Liu et al. fine-tuned their model based on Hill's Helping Skills Theory, introducing an emotion dataset labeled with support strategies to endow the dialogue system with preliminary emotional support capabilities.

[0005] Subsequently, research on psychological counseling systems and models mainly focused on the following three directions: (1) Generation and quality improvement of psychological counseling datasets: generating high-quality simulated datasets by collecting question-and-answer data from psychological counseling websites or using large language models. (2) Selection of psychological counseling strategies: analyzing the dialogue patterns of human psychological counselors and effective emotional support strategies, and developing automated strategy selection methods. (3) Use of stronger base models: adopting base models with larger parameter scales and stronger language comprehension capabilities to better capture complex emotional cues and dialogue details.

[0006] Nevertheless, previous research has largely neglected enhancing the personalized empathy capabilities of models, which are crucial for improving the effectiveness of student psychological counseling and form the cornerstone of psychotherapy. Existing psychological counseling systems and models have significant limitations in empathy capabilities, failing to effectively trigger deep emotional resonance in students, thus weakening their overall support effectiveness and student user experience. Therefore, enhancing the empathy capabilities of dialogue systems, particularly by leveraging professional empathy theory, is a key development direction for future AI applications in student mental health. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing mental health counseling methods or systems in terms of emotional support and empathy, and to provide a personalized student mental health counseling method and system based on a grouped hybrid expert model, so as to improve the empathy and personalization level of mental health counseling.

[0008] The inventive concept of this invention is based on the psychological theory of empathy. It dynamically selects appropriate counselor roles and adjusts support strategies in real time according to the student's emotions and background. Through a decoupled expert hybrid model (MoE) and experience retrieval technology, this invention ensures personalized and in-depth emotional support in every conversation. This invention provides an intelligent solution for emotional resonance in psychological counseling, enhancing the effectiveness and experience of mental health services.

[0009] To achieve the above-mentioned objectives, the present invention specifically adopts the following technical solution:

[0010] In a first aspect, the present invention provides a personalized student psychological counseling method based on a grouped hybrid expert model, which includes the following steps:

[0011] S1: Obtain student psychological counseling data in single-round question-and-answer dialogues, use a large language model to rewrite the single-round question-and-answer dialogue text into multi-round question-and-answer dialogue text according to preset prompt words, and embed counselor role experience and language style to generate structured dialogue text data.

[0012] S2: Extract students' question background information and counselor role labels from the dialogue text data to form a role matching dataset, and train the role matching model on the role matching dataset.

[0013] S3: Extract supporting policy labels, students' question background information, and contextual dialogue between students and counselors from the dialogue text data to form a policy prediction dataset, and train a policy prediction model on the policy prediction dataset.

[0014] S4: Train a grouped hybrid expert model on dialogue text data. The grouped hybrid expert model is obtained by replacing the feedforward neural network layer in the Transformer block of the LLaMA model with a grouped hybrid expert layer. In the grouped hybrid expert layer, a gating network is used to route consultant role information to the role expert group and support policy information to the policy expert group. Each of the two expert groups learns specific semantic features and controls the parameter scale through a low-rank method.

[0015] S5: Obtain representative experience texts for each counselor role, and use a large language model to embed emotion vectors into each representative experience text to generate emotion embedding vectors. Combine the emotion embedding vectors with the representative experience texts to form key-value pairs, creating a role experience database for retrieval.

[0016] S6: The student client takes the background information of the student's problem as input text, inputs the input text into the trained role matching model to predict the counselor role label, inputs the input text into the trained strategy prediction model to predict the support strategy label, encodes the two predicted labels and inputs them into the grouped hybrid expert model to generate an initial response, retrieves the most relevant representative experience text in the role experience library and fills it into the initial response as supplementary prompts, generates the final counseling response and returns it to the student client.

[0017] Based on the above scheme, each step can be implemented in the following preferred manner.

[0018] As a preferred embodiment of the first aspect mentioned above, the specific process of step S1 is as follows: First, collect a preset number of student psychological counseling data in single-round question-and-answer dialogues from an online psychological counseling website, based on N psychological counseling topics and N counselor roles, with each counselor role corresponding to each psychological counseling topic; then, convert the single-round question-and-answer dialogue text into multi-round dialogue question-and-answer text using a large language model with preset prompt words, batch insert counselor role experiences and embed role-specific language styles into the multi-round dialogue text, generating dialogue text data containing the quadruple of <student's question background information, counselor role label, support strategy label, context dialogue>, thus forming a dialogue text dataset; wherein, the question background information text of each student is composed of the student's question, detailed description, and topic keywords.

[0019] As a preferred embodiment of the first aspect mentioned above, in step S2, during the training process of the role matching model, the student's question background information text is encoded and used as input. The role matching model outputs the predicted counselor role, and the parameters of the role matching model are adjusted by an optimization algorithm to minimize the difference between the predicted counselor role and the counselor role label, so as to finally obtain the trained role matching model.

[0020] As a preferred embodiment of the first aspect mentioned above, in step S3, after forming the background information text of each student's question during the training process of the strategy prediction model, the text is concatenated with the contextual dialogue text between the student and the counselor to obtain the corresponding response strategy text for the student. Then, the response strategy text is encoded and used as input. The strategy prediction model outputs the predicted support strategy. The parameters of the strategy prediction model are adjusted by an optimization algorithm to minimize the difference between the predicted support strategy and the support strategy label, and finally, the trained strategy prediction model is obtained.

[0021] As a preferred embodiment of the first aspect mentioned above, in step S4, the response strategy text is encoded using an embedding layer, converting it from a natural language representation into a vector representation of a uniform dimension to form text input data, which serves as the input to the first Transformer block of the grouped hybrid expert model.

[0022] As a preferred embodiment of the first aspect mentioned above, in step S4, the role expert group contains N role experts, the strategy expert group contains M strategy experts, each role expert or strategy expert corresponds to a pair of low-rank matrices, each expert group corresponds to a gating network, the gating network corresponding to the role expert group is the first gating network, and the gating network corresponding to the strategy expert group is the second gating network.

[0023] As a preferred embodiment of the first aspect above, the specific processing flow in the grouped hybrid expert layer described in step S4 is as follows:

[0024] S41: Encode all predefined consultant roles and all support strategies into corresponding low-dimensional vectors using a pre-trained language model, forming role embedding matrices and strategy embedding matrices respectively; where each row of the role embedding matrix corresponds to a role embedding vector, and each row of the strategy embedding matrix corresponds to a strategy embedding vector.

[0025] S42: Input the role embedding matrix into the first gating network and output the activation weight of each role expert; input the strategy embedding matrix into the second gating network and output the activation weight of each strategy expert.

[0026] S43: Perform low-rank decomposition on the parameter matrix that needs to be trained for each expert to obtain two low-rank matrices. Use the ratio of the scaling factor to the dimension of the low-rank matrix as the low-rank coefficient. Multiply the two low-rank matrices corresponding to each expert with the low-rank coefficient to form the simulation parameter matrix corresponding to each expert.

[0027] S44: Multiply the simulation parameter matrix of each role expert by the input hidden state to obtain the output of each role expert. Based on the activation weights of the role experts, sum the outputs of all role experts to obtain the role representation output by the role expert group. Multiply the simulation parameter matrix of each policy expert by the input hidden state to obtain the output of each policy expert. Based on the activation weights of the policy experts, sum the outputs of all policy experts to obtain the policy representation output by the policy expert group.

[0028] S45: Multiply the pre-trained parameter matrix with the input hidden state to obtain the weighted hidden state. Add the weighted hidden state, role representation, and policy representation together to obtain the hidden state output by the grouped hybrid expert layer.

[0029] As a preferred embodiment of the first aspect, in the first gating network, each role embedding vector is multiplied by the gating parameter matrix of the first gating network and then normalized by the Softmax function to generate the activation weight of each role expert; in the second gating network, each policy embedding vector is multiplied by the gating parameter matrix of the second gating network and then normalized by the Softmax function to generate the activation weight of each policy expert.

[0030] As a preferred embodiment of the first aspect mentioned above, in step S6, the specific process of retrieving the most relevant representative experience text to the input text is as follows: after the input text is encoded by a large language model, an emotion vector representation is formed. The cosine similarity between the emotion vector representation and each emotion embedding vector in the role experience database is calculated, and the representative experience text with the highest cosine similarity is used as supplementary prompt information.

[0031] Secondly, the present invention provides a personalized student psychological counseling system based on a grouped hybrid expert model, comprising:

[0032] The data acquisition module is used to acquire student psychological counseling data in single-round question-and-answer dialogues. It uses a large language model to rewrite the single-round question-and-answer dialogue text into multi-round question-and-answer dialogue text according to preset prompt words, and embeds the counselor's role experience and language style to generate structured dialogue text data.

[0033] The first model training module is used to extract students' question background information and counselor role labels from dialogue text data to form a role matching dataset, and to train the role matching model on the role matching dataset.

[0034] The second model training module is used to extract supporting policy labels, students' question background information, and contextual dialogue between students and counselors from the dialogue text data to form a policy prediction dataset, and to train the policy prediction model on the policy prediction dataset.

[0035] The third model training module is used to train a grouped hybrid expert model on dialogue text data. The grouped hybrid expert model is obtained by replacing the feedforward neural network layer in the Transformer block of the LLaMA model with a grouped hybrid expert layer. In the grouped hybrid expert layer, a gating network is used to route consultant role information to the role expert group and support policy information to the policy expert group. Each of the two expert groups learns specific semantic features and controls the parameter scale through a low-rank method.

[0036] The role experience library construction module is used to obtain representative experience texts for each counselor role, and use a large language model to embed emotion vectors into each representative experience text to generate emotion embedding vectors. The emotion embedding vectors are then combined with the representative experience texts to form key-value pairs, thus forming a role experience library for retrieval.

[0037] The consultation response generation module is used to input the background information of the student who needs psychological counseling into the student client. The input text is fed into a trained role matching model to predict counselor role labels and into a trained strategy prediction model to predict supporting strategy labels. The two predicted labels are encoded and then fed into a grouped hybrid expert model to generate an initial response. The module retrieves the most relevant representative experience text from the role experience database and fills it into the initial response as supplementary prompts. Finally, the consultation response is generated and returned to the student client.

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] This invention addresses the shortcomings of existing mental health counseling systems in terms of emotional support and empathy capabilities by proposing a personalized student psychological counseling method based on a grouped hybrid expert model. This method is grounded in counselor role selection and empathic experience retrieval. By dynamically selecting appropriate counselor roles and retrieving relevant empathic experiences based on the student's emotions and background, this invention provides deep emotional resonance during the counseling process. This innovation enables the psychological counseling method to adjust support strategies in real time, ensuring effective assistance in addressing diverse emotional needs. By introducing counselor roles and combining them with multi-dimensional emotional support strategies, this invention's psychological counseling method can provide users with stable, personalized, and efficient emotional support even with limited emotional data and role settings. This expands its application scenarios in practical mental health services and significantly improves the effectiveness of psychological counseling services and the user experience. Attached Figure Description

[0040] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0041] Figure 2 A schematic diagram of the architecture of a Transformer block and a grouped hybrid expert layer provided in an embodiment of the present invention;

[0042] Figure 3 This is a system block diagram of the present invention. Detailed Implementation

[0043] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0044] In the description of this invention, it should be understood that the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature.

[0045] To address the shortcomings of existing technologies in emotional support and empathy, this invention provides a personalized student psychological counseling method based on a grouped hybrid expert model. This method dynamically adjusts by combining multi-dimensional empathy theory and emotional support strategies, trains on psychological counseling data, and extracts emotionally resonant features from students' emotional changes and background information. Empathic experiences are inserted through experience retrieval, thereby enhancing the empathy capability of the psychological counseling method. In this way, this invention can provide stable and effective emotional support in different contexts, significantly improving the effectiveness of mental health services and the user experience.

[0046] like Figure 1 As shown, in a preferred embodiment of the present invention, the personalized student psychological counseling method based on a grouped hybrid expert model includes the following steps S1 to S6. The specific implementation process of each step will be described in detail below.

[0047] S1: Obtain student psychological counseling data in single-round question-and-answer dialogues, use a large language model to rewrite the single-round question-and-answer dialogue text into multi-round question-and-answer dialogue text according to preset prompts, and embed counselor role experience and language style to generate structured dialogue text data.

[0048] It should be noted that, in this invention, the specific process of step S1 is as follows: First, a preset number of student psychological counseling data in single-turn question-and-answer dialogues are collected from an online psychological counseling website, based on N psychological counseling topics and N counselor roles, with each counselor role corresponding to each psychological counseling topic; then, the single-turn question-and-answer dialogue text is converted into multi-turn dialogue question-and-answer text using a preset prompt word prompting large language model (GPT-4o), and counselor role experiences are batch-inserted into the multi-turn dialogue text, embedding the role's specific language style, generating dialogue text data containing the quadruple of <student's problem background information, counselor role label, support strategy label, context dialogue>, thereby forming a dialogue text dataset.

[0049] It should be noted that, in this embodiment, the question of the m-th student is... Detailed description and topic keywords When pieced together, this forms the background information text of the student's problem. , represented as:

[0050]

[0051] In this context, ; indicates concatenated text; [Question] represents the student's question text; [Description] represents the student's detailed description text; and [Keywords] represents the student's topic keywords text.

[0052] In this embodiment, the dialogue text dataset is divided into a training set and a test set in a 9:1 ratio for subsequent model training and evaluation.

[0053] In this embodiment, the support strategy refers to dividing psychological counseling into three stages: exploration, comforting, and action. Each stage uses different response strategies, such as questioning, emotional response, paraphrasing or interpretation, self-disclosure, encouragement and reassurance, providing information, providing advice, and other strategies, totaling eight strategies. These eight strategies are used in combination to complete the psychological counseling dialogue.

[0054] S2: Extract students' question background information and counselor role labels from the dialogue text data to form a role matching dataset, and train the role matching model on the role matching dataset.

[0055] It should be noted that in step S2 of this embodiment, during the training process of the role matching model, the student's question background information text is encoded and used as input. The role matching model outputs the predicted counselor role. The parameters of the role matching model are adjusted by the optimization algorithm to minimize the difference between the predicted counselor role and the counselor role label, and finally the trained role matching model is obtained.

[0056] It should be noted that in step S2 of this embodiment, the dialogue text dataset in S1 is preprocessed to retain the student's personal background and corresponding counselor role labels, and the tuple <student's problem background information, counselor role label> is extracted to construct a role matching dataset. Then, based on the relevant experience and personality traits of each counselor role, the most suitable counselor role is manually assigned to each piece of problem background information as the golden role label. During training, the textual background information of the student's questions is encoded and input into the role matching model. The golden role label is used as the real label. The model is trained to learn the relationship between the student's background information and the golden role label. Thus, during the inference phase, the model can automatically recommend a suitable consultant role based on the student's background information. .

[0057] In this invention, the role matching model can be selected by those skilled in the art according to actual needs. In this embodiment, the role matching model adopts the pre-trained language model RoBERTa (Robustly Optimized BERT Approach), the implementation of which is prior art and will not be described in detail here.

[0058] S3: Extract supporting policy labels, students' question background information, and contextual dialogue between students and counselors from the dialogue text data to form a policy prediction dataset, and train a policy prediction model on the policy prediction dataset.

[0059] It should be noted that in step S3 of this embodiment, after the background information text of the m-th student's question is generated during the training process of the strategy prediction model, it is compared with the contextual dialogue text between the student and the counselor. By concatenating the text, we obtain the corresponding response strategy text for the student. The response policy text is then encoded and used as input. The policy prediction model outputs the predicted supporting policy. An optimization algorithm is used to adjust the parameters of the policy prediction model to minimize the difference between the predicted supporting policy and the supporting policy label, ultimately obtaining a trained policy prediction model. The above response policy text can be represented as:

[0060]

[0061] Here, [Context] represents the contextual dialogue text between the student and the counselor.

[0062] It should be noted that in step S3 of this embodiment, after determining the consultant's role, the text of the student's question, the consultant's response, and their corresponding strategy tags are retained. The <support strategy tag, context dialogue> tuple is extracted to construct a strategy prediction dataset for training the strategy prediction model. Then, the most suitable consultation strategy (support strategy) is manually assigned as the gold strategy tag for this consultation scenario. During training, the encoded response strategy text is input into the strategy prediction model, using the golden policy label as the true label. The model is trained to learn the relationship between the response strategy text and the golden policy label. Thus, during the inference phase, the model can recommend an appropriate support strategy based on the student's problem background information and the content of the conversation with the counselor. .

[0063] In this invention, the policy prediction model can be selected by those skilled in the art according to actual needs. In this embodiment, the policy prediction model adopts the pre-trained language model RoBERTa (Robustly Optimized BERT Approach), the implementation of which is prior art and will not be described in detail here.

[0064] S4: Train a grouped hybrid expert model on dialogue text data. The grouped hybrid expert model is obtained by replacing the feed-forward network (FFN) layer in the Transformer block of the LLaMA (Large Language Model Meta AI) model with a grouped hybrid expert layer, while other parts of the Transformer block (such as attention layer, residual connection and normalization layer) remain unchanged. In the grouped hybrid expert layer, a gating network is used to route consultant role information to the role expert group and support policy information to the policy expert group. Each expert group learns specific semantic features and controls the parameter scale through a low-rank method.

[0065] It should be noted that in step S4 of this invention, the response strategy text is encoded using an embedding layer, converting it from a natural language representation into a vector representation of a unified dimension to form text input data, which serves as the input to the first Transformer block of the grouped hybrid expert model.

[0066] It should be noted that in step S4 of this invention, the role expert group contains N role experts, the strategy expert group contains M strategy experts, each role expert or strategy expert corresponds to a pair of low-rank matrices, each expert group corresponds to a gating network, the gating network corresponding to the role expert group is the first gating network, and the gating network corresponding to the strategy expert group is the second gating network.

[0067] In step S4 of this embodiment, the internal structure of the grouped hybrid expert layer is as follows: Figure 2 As shown, its core design involves the introduction of two independent expert groups ( Figure 2 Group 1 and Figure 2 Group 1 corresponds to the Role Group, used to model various counselor roles in psychological counseling, containing N role experts, each handling the semantic features of their corresponding role; Group 2 corresponds to the Strategy Group, used to model support strategies, containing M strategy experts, each handling the semantic features of their corresponding strategy. Each expert corresponds to a pair of low-rank matrices (…). Figure 2 In and , and … and , and This enables efficient representation and updating of parameters. Furthermore, each expert group is equipped with an independent gating network to determine the degree of matching between the input information and each expert, and to dynamically select experts and assign activation weights based on student input.

[0068] It should be noted that the specific processing flow in the grouped hybrid expert layer described in step S4 of this invention is as follows:

[0069] S41: Encode all predefined consultant roles (N in total) and all support strategies (M in total) into corresponding low-dimensional vectors using a pre-trained language model (such as BERT), forming role embedding matrices respectively. and policy embedding matrix .

[0070] In this embodiment, each row of the role embedding matrix corresponds to a role embedding vector, and the matrix consists of N role embedding vectors, each representing a global semantic representation of a role. Each row of the policy embedding matrix corresponds to a policy embedding vector, and the matrix consists of M policy embedding vectors, each representing a global semantic representation of a policy. These embedding vectors will be used in the gating network to activate the corresponding experts, thereby achieving dynamic modeling of the role-policy dimension.

[0071] Specifically, the first Embedded vectors of each role With the each policy embedding vector It is expressed as follows:

[0072]

[0073] Where d represents the dimension of the role embedding vector or strategy embedding vector.

[0074] S42: Input the role embedding matrix into the first gating network and output the activation weight of each role expert. Input the policy embedding matrix into the second gating network and output the activation weight of each policy expert.

[0075] In the first gating network, each role embedding vector is multiplied by the gating parameter matrix of the first gating network and then normalized by the Softmax function to generate the activation weight of each role expert; in the second gating network, each policy embedding vector is multiplied by the gating parameter matrix of the second gating network and then normalized by the Softmax function to generate the activation weight of each policy expert.

[0076] In step S42 of this embodiment, the input to the gating network is a role embedding vector or a policy embedding vector. The activation weight (i.e., contribution level) of each expert is calculated through a set of learnable gating weight matrices. The higher the activation weight, the greater the contribution of the expert to the current input. The obtained activation weights are then processed by the Softmax function to ensure that the sum of all activation weights is 1, so that the model has selectivity.

[0077] No. Embedded vectors of each role Corresponding activation weight With the each policy embedding vector Corresponding activation weight It can be represented as:

[0078]

[0079] in, This represents the gating parameter matrix of the first gating network; This represents the gating parameter matrix of the first gating network.

[0080] S43: Perform low-rank decomposition on the parameter matrix that needs to be trained for each expert to obtain two low-rank matrices. Use the ratio of the scaling factor to the dimension of the low-rank matrix as the low-rank coefficient. Multiply the two low-rank matrices corresponding to each expert with the low-rank coefficient to form the simulation parameter matrix corresponding to each expert.

[0081] In step S43 of this embodiment, to achieve efficient parameter updating and model lightweighting, the LoRA (Low-Rank Adaptation) mechanism is introduced to perform low-rank decomposition on the parameter matrix of each expert. Here, role experts and policy experts perform the same operation, so there is no distinction. Taking the k-th expert (which can be either a role expert or a policy expert) as an example, this expert does not directly update the pre-trained parameter matrix. Instead, two low-rank matrices are used. and To simulate parameter updates: This decomposes the high-dimensional embedding into multiple low-rank matrices for parameter constraints. This represents the matrix of simulated parameters that need to be trained; is the scaling factor, and r is the dimension of the low-rank matrix.

[0082] S44: Multiply the simulation parameter matrix of each role expert by the input hidden state x to obtain the output of each role expert. Based on the activation weights of the role experts, sum the outputs of all role experts to obtain the role representation output by the role expert group. The simulation parameter matrix of each policy expert is multiplied by the input hidden state x to obtain the output of each policy expert. The outputs of all policy experts are then weighted and summed based on their activation weights to obtain the policy representation output by the policy expert group. :

[0083]

[0084]

[0085] In step S44 of this embodiment, the input hidden state x is the layer before the grouped hybrid expert layer ( Figure 2 The output of the root mean square normalization layer in the middle.

[0086] S45: Multiply the pre-trained parameter matrix by the input hidden state to obtain the weighted hidden state. Add the weighted hidden state, role representation, and policy representation together to obtain the hidden state output by the grouped hybrid expert layer. :

[0087]

[0088] S5: Obtain representative experience texts for each counselor role, and use a large language model to embed emotion vectors into each representative experience text to generate emotion embedding vectors. Combine the emotion embedding vectors with the representative experience texts to form key-value pairs, creating a role experience database for retrieval.

[0089] It should be noted that step S5 of this invention designs a role-based experience retrieval mechanism to construct a role experience database. Specifically, it first obtains representative experience texts corresponding to each counselor role. These representative experience texts can be plot materials, script excerpts, or film / television character content related to the counselor role's background. A representative experience text of a consultant role is recorded as follows: Subsequently, each representative experience text was encoded into a corresponding sentiment embedding vector using a large language model. A representative experience text of a consultant role is recorded as follows: The emotional embedding vectors belonging to the same character are combined with representative experience texts to form key-value pairs. This ultimately forms a role experience database for retrieval. .

[0090] S6: The student client takes the background information of the student's problem that needs psychological counseling as input text, inputs the input text into the trained role matching model to predict the counselor role label, inputs the input text into the trained strategy prediction model to predict the support strategy label, encodes the two predicted labels and inputs them into the grouped hybrid expert model to generate an initial response, retrieves the most relevant representative experience text in the role experience library and fills it into the initial response as supplementary prompts, generates the final counseling response and returns it to the student client.

[0091] It should be noted that, in step S6 of the present invention, the specific process of retrieving the most relevant representative experience text to the input text is as follows: after the input text is encoded by a large language model, an emotion vector representation is formed. The cosine similarity between the emotion vector representation and each emotion embedding vector in the role experience database is calculated. The representative experience text with the highest cosine similarity is used as supplementary prompt information.

[0092] In this embodiment, the retrieved representative experience text is added as supplementary prompts to the initial response generated by the model, guiding the model to generate more empathetic and context-appropriate response content, thereby forming the final consultation response.

[0093] The personalized student psychological counseling method based on a grouped hybrid expert model described in the above embodiments will be applied to a multi-role, multi-strategy dataset for evaluation index testing. The specific steps are as described in S1-S6 and will not be repeated here; the main focus is on demonstrating the specific parameters and technical effects.

[0094] Example

[0095] This embodiment follows the implementation process of steps S1 to S6 described above. First, real single-round question-and-answer psychological counseling data is collected. Multi-role, multi-strategy psychological counseling datasets are obtained through preprocessing using a large language model. These data are used as training data, and the model of this invention is trained with randomly set seeds and its performance is tested on a test set. During the evaluation process, automation metrics in the natural language processing domain (BLEU-1, ROUGE-L) and large language model evaluation metrics (Skillfulness, Empathy, Helpfulness) are used, with GPT-4o-2024-11-20 used to evaluate the large language model. The comparison results of the method of this invention with other models are shown in Table 1. The implementation methods of the MeChat model, SoulChat model, CPsyCounX model, and CharCounselor model in Table 1 are existing technologies and will not be described further.

[0096] Table 1. Evaluation Form for the Effectiveness of Psychological Counseling

[0097]

[0098] This embodiment also demonstrates the visualization effect of the present invention in a real psychological counseling scenario, as shown below:

[0099] Visiting student: I've been feeling really down lately. I can't sleep at night, and I sleep all day. I'm wondering if it's because of the pressure of the high school entrance exam? Since last week, I've felt increasingly unwilling to try anymore.

[0100] Counselor: [Question] I'm Lao Sun, and you seem quite down. Come on, tell me, what's been going on lately?

[0101] Visiting student: I feel like I've had suicidal thoughts for a long time, but they've become especially strong recently. I have no interest in anything, and sometimes I can't help but cry, and I even try to hurt myself.

[0102] Counselor: [Emotional Response] Listening to your pain, it feels like a mountain is pressing down on my heart, making it hard to breathe. It must be incredibly painful for you, right? And the anxiety and helplessness you're experiencing are truly suffocating. This kind of torment is just like how I felt when I was pressed under the Five Elements Mountain back then.

[0103] Visiting student: Yes, I feel like I'll be free after the exams. I'm really afraid of failing, afraid of disappointing my family, and even afraid of being called dramatic when I hurt myself.

[0104] Counselor: [Restatement or Explanation] I, Lao Sun, finally understand. Your pressure probably comes from the bondage of "fear of losing" and "fear of what people will say," right? It's making you feel suffocated. Did I guess right?

[0105] Visiting student: Yes, I often try to work hard, but then I feel that these efforts are in vain. In short, nothing makes me happy, and all I can do is try not to completely break down.

[0106] Counselor: [Self-disclosure] Wow! You managed to hold on under that immense pressure; you're incredibly strong! It's like when I was on my journey to the West, facing Red Boy. That little demon almost singed all my monkey hair with his flames, and I also had thoughts of giving up! But in the end, I, Old Sun, gritted my teeth and overcame the difficulty through sheer willpower! You are doing something amazing right now, really, don't underestimate yourself! You are already very strong.

[0107] Visiting student: Thank you for saying all this, but I still can't control the urge to cry. Every time I cry, it feels like I'm trying to release something, but nothing changes afterward.

[0108] Counselor: [Offering advice] Crying is human nature. It's like when I, Old Sun, wield my staff to fight demons—it's all about releasing all my energy. But you might try going for a run, or playing basketball for a bit. Moving around might help you, like me, relieve the pressure in your heart!

[0109] Visiting student: Thank you for your suggestion. I didn't expect there could be such specific and easy steps to try. I hope it really works.

[0110] Counselor: [Encouragement and reassurance] You're already very strong to take this step! Take it one step at a time, don't push yourself too hard. You might find that every step you take has been very rewarding! If you have any questions, come find me, Lao Sun!

[0111] The above text dialogue demonstrates how the method of this invention automatically matches the most suitable counselor role (such as "Sun Wukong") based on the student's background information (mainly growth-related issues) and generates corresponding counseling strategy suggestions (marked in square brackets) in conjunction with the dialogue context. When the method determines that it needs to incorporate the role's empathic experience, it retrieves the most relevant representative experience text from the role's experience database through emotion embedding vectors and inserts it into the dialogue in text form to enhance empathic expression. Therefore, this invention demonstrates a strong ability to create personalized emotional resonance in personalized psychological counseling tasks through a synergistic mechanism of role matching, strategy prediction, and experience embedding.

[0112] In summary, the role matching model of this invention can match corresponding counselor roles based on students' problem background information (such as age, gender, problem type, etc.). The matching of counselor roles and problem background information is based on predefined roles and problem types, such as "Sun Wukong" matching "growth" problems, "Juliet" matching "love" problems, etc. This matching relationship will determine which role experts the model will call for response. The strategy prediction model can adopt psychological counseling strategies (such as empathic listening, cognitive restructuring, action guidance, etc.) based on the current student's problem status and role setting. When a student needs psychological counseling, this strategy information will be passed as a guidance signal to the mixed expert model, thereby generating an initial response with appropriate psychological intervention intentions. A mixed expert layer is designed in the mixed expert model, which adopts a grouped MoE (Mixture of Experts) structure. Two expert groups handle role and strategy outputs respectively, with different experts within each group representing different roles or strategies. A gating network dynamically regulates the outputs of multiple experts, acting as a "gate" to adaptively control the activation level of each expert (e.g., role, strategy) based on the user's current input context. This enhances the model's expressive and reasoning abilities in different contexts, thereby enabling personalized and diverse response generation. A role experience database is constructed, and by calculating the emotional similarity between the student and multiple counselor role experiences in the database, the most relatable representative experience texts are retrieved as contextual cues to help the model better understand the student's emotional state.

[0113] It should also be noted that the personalized student psychological counseling method based on a grouped hybrid expert model in the above embodiments can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a personalized student psychological counseling system based on a grouped hybrid expert model, corresponding to the personalized student psychological counseling method based on a grouped hybrid expert model provided in the above embodiments, such as... Figure 3 As shown, it includes:

[0114] The data acquisition module is used to acquire student psychological counseling data in single-round question-and-answer dialogues. It uses a large language model to rewrite the single-round question-and-answer dialogue text into multi-round question-and-answer dialogue text according to preset prompt words, and embeds the counselor's role experience and language style to generate structured dialogue text data.

[0115] The first model training module is used to extract students' question background information and counselor role labels from dialogue text data to form a role matching dataset, and to train the role matching model on the role matching dataset.

[0116] The second model training module is used to extract supporting policy labels, students' question background information, and contextual dialogue between students and counselors from the dialogue text data to form a policy prediction dataset, and to train the policy prediction model on the policy prediction dataset.

[0117] The third model training module is used to train a grouped hybrid expert model on dialogue text data. The grouped hybrid expert model is obtained by replacing the feedforward neural network layer in the Transformer block of the LLaMA model with a grouped hybrid expert layer. In the grouped hybrid expert layer, a gating network is used to route consultant role information to the role expert group and support policy information to the policy expert group. Each of the two expert groups learns specific semantic features and controls the parameter scale through a low-rank method.

[0118] The role experience library construction module is used to obtain representative experience texts for each counselor role, and use a large language model to embed emotion vectors into each representative experience text to generate emotion embedding vectors. The emotion embedding vectors are then combined with the representative experience texts to form key-value pairs, thus forming a role experience library for retrieval.

[0119] The consultation response generation module is used to input the background information of the student who needs psychological counseling into the student client. The input text is fed into a trained role matching model to predict counselor role labels and into a trained strategy prediction model to predict supporting strategy labels. The two predicted labels are encoded and then fed into a grouped hybrid expert model to generate an initial response. The module retrieves the most relevant representative experience text from the role experience database and fills it into the initial response as supplementary prompts. Finally, the consultation response is generated and returned to the student client.

[0120] It should also be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the embodiments provided in this application, the division of steps or modules in the system and method is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or steps may be combined or integrated together, and a module or step may also be split.

[0121] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A personalized student psychological counseling method based on a grouped hybrid expert model, characterized in that, Includes the following steps: S1: Obtain student psychological counseling data in single-round question-and-answer dialogues, use a large language model to rewrite the single-round question-and-answer dialogue text into multi-round question-and-answer dialogue text according to preset prompt words, and embed counselor role experience and language style to generate structured dialogue text data. S2: Extract students' question background information and counselor role labels from the dialogue text data to form a role matching dataset, and train the role matching model on the role matching dataset. S3: Extract supporting policy labels, students' question background information, and contextual dialogue between students and counselors from the dialogue text data to form a policy prediction dataset, and train a policy prediction model on the policy prediction dataset. S4: Train a grouped hybrid expert model on dialogue text data. The grouped hybrid expert model is obtained by replacing the feedforward neural network layer in the Transformer block of the LLaMA model with a grouped hybrid expert layer. In the grouped hybrid expert layer, a gating network is used to route consultant role information to the role expert group and support policy information to the policy expert group. Each of the two expert groups learns specific semantic features and controls the parameter scale through a low-rank method. S5: Obtain representative experience texts for each counselor role, and use a large language model to embed emotion vectors into each representative experience text to generate emotion embedding vectors. Combine the emotion embedding vectors with the representative experience texts to form key-value pairs, creating a role experience database for retrieval. S6: The student client takes the background information of the student's problem that needs psychological counseling as input text, inputs the input text into the trained role matching model to predict the counselor role label, inputs the input text into the trained strategy prediction model to predict the support strategy label, encodes the two predicted labels and inputs them into the grouped hybrid expert model to generate an initial response, retrieves the most relevant representative experience text in the role experience library and fills it into the initial response as supplementary prompts, generates the final counseling response and returns it to the student client.

2. The personalized student psychological counseling method based on a grouped hybrid expert model as described in claim 1, characterized in that, The specific process of step S1 is as follows: First, collect a preset number of student psychological counseling data from the online psychological counseling website, based on N psychological counseling topics and N counselor roles, with each counselor role corresponding to each psychological counseling topic. Subsequently, the single-turn question-and-answer dialogue text is transformed into multi-turn dialogue question-and-answer text using a large language model with preset prompt words. Consultant role experiences are then inserted in batches into the multi-turn dialogue text, and role-specific language styles are embedded to generate dialogue text data containing the quadruple <student's question background information, consultant role label, support strategy label, context dialogue>, thus forming a dialogue text dataset. Among them, the question background information text for each student is composed of the student's question, detailed description, and topic keywords.

3. The personalized student psychological counseling method based on a grouped hybrid expert model as described in claim 1, characterized in that, In step S2, during the training of the role matching model, the student's question background information text is encoded and used as input. The role matching model outputs the predicted counselor role. The parameters of the role matching model are adjusted by an optimization algorithm to minimize the difference between the predicted counselor role and the counselor role label, and finally a well-trained role matching model is obtained.

4. The personalized student psychological counseling method based on a grouped hybrid expert model as described in claim 1, characterized in that, In step S3, during the training of the policy prediction model, after forming the background information text of each student's question, it is concatenated with the contextual dialogue text between the student and the counselor to obtain the corresponding response strategy text for the student. Then, the response strategy text is encoded and used as input. The policy prediction model outputs the predicted support strategy. The parameters of the policy prediction model are adjusted by an optimization algorithm to minimize the difference between the predicted support strategy and the support strategy label, and finally, the trained policy prediction model is obtained.

5. The personalized student psychological counseling method based on a grouped hybrid expert model as described in claim 4, characterized in that, In step S4, the response strategy text is encoded using an embedding layer, converting it from a natural language representation into a vector representation of a uniform dimension to form text input data, which serves as the input to the first Transformer block of the grouped hybrid expert model.

6. The personalized student psychological counseling method based on a grouped hybrid expert model as described in claim 1, characterized in that, In step S4, the role expert group contains N role experts and the strategy expert group contains M strategy experts. Each role expert or strategy expert corresponds to a pair of low-rank matrices. Each expert group corresponds to a gating network. The gating network corresponding to the role expert group is the first gating network, and the gating network corresponding to the strategy expert group is the second gating network.

7. The personalized student psychological counseling method based on a grouped hybrid expert model as described in claim 6, characterized in that, The specific processing flow in the grouped hybrid expert layer described in step S4 is as follows: S41: Encode all predefined consultant roles and all support strategies into corresponding low-dimensional vectors using a pre-trained language model, forming role embedding matrices and strategy embedding matrices respectively; where each row of the role embedding matrix corresponds to a role embedding vector, and each row of the strategy embedding matrix corresponds to a strategy embedding vector. S42: Input the role embedding matrix into the first gating network and output the activation weight of each role expert; input the strategy embedding matrix into the second gating network and output the activation weight of each strategy expert. S43: Perform low-rank decomposition on the parameter matrix that needs to be trained for each expert to obtain two low-rank matrices. Use the ratio of the scaling factor to the dimension of the low-rank matrix as the low-rank coefficient. Multiply the two low-rank matrices corresponding to each expert with the low-rank coefficient to form the simulation parameter matrix corresponding to each expert. S44: Multiply the simulation parameter matrix of each role expert by the input hidden state to obtain the output of each role expert. Based on the activation weights of the role experts, sum the outputs of all role experts to obtain the role representation output by the role expert group. Multiply the simulation parameter matrix of each policy expert by the input hidden state to obtain the output of each policy expert. Based on the activation weights of the policy experts, sum the outputs of all policy experts to obtain the policy representation output by the policy expert group. S45: Multiply the pre-trained parameter matrix with the input hidden state to obtain the weighted hidden state. Add the weighted hidden state, role representation, and policy representation together to obtain the hidden state output by the grouped hybrid expert layer.

8. The personalized student psychological counseling method based on a grouped hybrid expert model as described in claim 7, characterized in that, In the first gating network, each role embedding vector is multiplied by the gating parameter matrix of the first gating network and then normalized by the Softmax function to generate the activation weight of each role expert; in the second gating network, each policy embedding vector is multiplied by the gating parameter matrix of the second gating network and then normalized by the Softmax function to generate the activation weight of each policy expert.

9. The personalized student psychological counseling method based on a grouped hybrid expert model as described in claim 1, characterized in that, In step S6, the specific process of retrieving the most relevant representative experience text to the input text is as follows: after the input text is encoded by a large language model, an emotion vector representation is formed. The cosine similarity between the emotion vector representation and each emotion embedding vector in the role experience database is calculated. The representative experience text with the highest cosine similarity is used as supplementary prompt information.

10. A personalized student psychological counseling system based on a grouped hybrid expert model, characterized in that, include: The data acquisition module is used to acquire student psychological counseling data in single-round question-and-answer dialogues. It uses a large language model to rewrite the single-round question-and-answer dialogue text into multi-round question-and-answer dialogue text according to preset prompt words, and embeds the counselor's role experience and language style to generate structured dialogue text data. The first model training module is used to extract students' question background information and counselor role labels from dialogue text data to form a role matching dataset, and to train the role matching model on the role matching dataset. The second model training module is used to extract supporting policy labels, students' question background information, and contextual dialogue between students and counselors from the dialogue text data to form a policy prediction dataset, and to train the policy prediction model on the policy prediction dataset. The third model training module is used to train a grouped hybrid expert model on dialogue text data. The grouped hybrid expert model is obtained by replacing the feedforward neural network layer in the Transformer block of the LLaMA model with a grouped hybrid expert layer. In the grouped hybrid expert layer, a gating network is used to route consultant role information to the role expert group and support policy information to the policy expert group. Each of the two expert groups learns specific semantic features and controls the parameter scale through a low-rank method. The role experience library construction module is used to obtain representative experience texts for each counselor role, and use a large language model to embed emotion vectors into each representative experience text to generate emotion embedding vectors. The emotion embedding vectors are then combined with the representative experience texts to form key-value pairs, thus forming a role experience library for retrieval. The consultation response generation module is used to input the background information of the student who needs psychological counseling into the student client. The input text is fed into a trained role matching model to predict counselor role labels and into a trained strategy prediction model to predict supporting strategy labels. The two predicted labels are encoded and then fed into a grouped hybrid expert model to generate an initial response. The module retrieves the most relevant representative experience text from the role experience database and fills it into the initial response as supplementary prompts. Finally, the consultation response is generated and returned to the student client.

Images