Personalized student psychological consultation method and system based on grouped mixed expert model

Abstract

The invention discloses a personalized student psychological consultation method and system based on a grouping mixed expert model, and belongs to the field of natural language processing. According to the method, firstly, single-round question and answer data are rewritten into multiple rounds of dialogues, consultant role features are embedded, and dialogue text data are formed; then, a problem background, a role label and a support strategy are extracted, and a role matching model and a strategy prediction model are trained respectively; a grouping hybrid expert model is constructed by improving an LLaMA model structure, and routing learning of role and strategy features is realized by using a gating network. In the reasoning stage, consultant roles and support strategy labels are determined through the two prediction models, preliminary replies are generated through the grouping and mixing expert model, supplementation is carried out in combination with related experiences retrieved from a role experience library which is constructed in advance, and finally personalized consultation replies are formed and returned to student clients. The method realizes organic combination of consultant role matching, support strategy prediction and emotional reply generation.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and in particular relates to a personalized student psychological consultation method and system based on a grouped hybrid expert model. Background Art

[0002] Mental health is an important component of overall human health, directly affecting an individual's sense of well-being, learning efficiency, and social adaptability. Since the beginning of the 21st century, with increasing academic pressure, fierce competition for admission to higher education, and a more complex social environment, mental health issues among students have become increasingly prominent. According to the World Mental Health Report released by the World Health Organization (WHO) in 2022, approximately one in eight people worldwide suffers from a mental disorder, and adolescents and students are among the high-risk groups. However, due to high counseling costs and a shortage of professional psychological resources on campus, student mental health services face severe challenges in terms of accessibility and coverage. There is an urgent need to use artificial intelligence (AI) technology to improve intervention efficiency and service accessibility.

[0003] Fortunately, the rapid development of Natural Language Processing (NLP) technology in recent years has provided a potential solution to this core problem. In particular, the rapid advancement of Large Language Models (LLMs), such as GPT-4, Claude-3.5, and LLaMA3, has achieved remarkable results in dialogue generation and emotional support, initially demonstrating its ability to provide psychological comfort and advice to students.

[0004] However, the response styles of these models generally tend to be neutral assistants, lacking distinctive language styles and deep empathy, making them unable to meet users' needs for emotional support. To address this shortcoming, some research has attempted to incorporate psychological counseling theory into dialogue systems. For example, Liu et al. fine-tuned a model based on Hill's Helping Skills Theory. By introducing an emotion dataset with support strategy labels, they endowed the dialogue system with preliminary emotional support capabilities.

[0005] Since then, research on psychological counseling systems and models has focused on the following three directions: (1) Generation and quality improvement of psychological counseling datasets: by collecting question-and-answer data from psychological counseling websites or using large language models to generate high-quality simulated datasets. (2) Selection of psychological counseling strategies: by analyzing the conversation patterns and effective emotional support strategies of human counselors and developing automated strategy selection methods. (3) Use of stronger basic models: by using basic models with larger parameter sizes and stronger language understanding capabilities to better capture complex emotional clues and conversation details.

[0006] Despite this, previous research has largely neglected enhancing models' personalized empathy capabilities, a crucial factor in improving the effectiveness of student counseling and a cornerstone of psychotherapy. Existing counseling systems and models exhibit significant limitations in empathy, making it difficult to effectively trigger students' deep emotional resonance, thereby weakening their overall support effectiveness and user experience. Therefore, enhancing the empathy capabilities of dialogue systems, particularly leveraging professional empathy theory, is a key development direction for future AI applications in student mental health. Summary of the Invention

[0007] The purpose of the present invention is to address the deficiencies of existing mental health counseling methods or systems in terms of emotional support and empathy, and to provide a personalized student psychological counseling method and system based on a grouped hybrid expert model to enhance the empathy and personalization level of mental health counseling.

[0008] The inventive concept of this invention is to dynamically select appropriate counselor roles based on psychological empathy theory and adjust support strategies in real time according to the student's emotions and background. By utilizing a decoupled mixture of experts (MoE) model and experience retrieval technology, this invention ensures personalized and in-depth emotional support in every conversation. This invention provides an intelligent solution for emotional resonance in psychological counseling, enhancing the effectiveness and experience of mental health services.

[0009] In order to achieve the above-mentioned object of the invention, the present invention specifically adopts the following technical solutions:

[0010] In a first aspect, the present invention provides a personalized student psychological counseling method based on a group hybrid expert model, which comprises the following steps:

[0011] S1: Obtain student psychological counseling data of single-round question-and-answer dialogues, use a large language model to rewrite the single-round question-and-answer dialogue text into a multi-round question-and-answer dialogue text according to preset prompt words, and embed the counselor's role experience and language style to generate structured dialogue text data;

[0012] S2: Extract students’ problem background information and counselor role labels from the conversation text data to form a role matching dataset, and train the role matching model on the role matching dataset;

[0013] S3: Extract support strategy labels, students’ problem background information, and contextual conversations between students and counselors from the conversation text data to form a strategy prediction dataset. Train the strategy prediction model on the strategy prediction dataset.

[0014] S4: A grouped mixture of experts model is trained on the conversational text data. The grouped mixture of experts model is obtained by replacing the feedforward neural network layer in the Transformer block of the LLaMA model with a grouped mixture of experts layer. In the grouped mixture of experts layer, a gating network is used to route the consultant role information to the role expert group and the support strategy information to the strategy expert group. The two expert groups each learn specific semantic features, and the parameter scale is controlled by a low-rank method.

[0015] S5: Obtain representative experience texts for each consultant role, and use a large language model to embed sentiment vectors for each representative experience text to generate sentiment embedding vectors. The sentiment embedding vectors and representative experience texts are combined into key-value pairs to form a role experience library for retrieval.

[0016] S6: The problem background information of the student who needs psychological counseling is used as input text through the student client, and the input text is input into the trained role matching model to predict the counselor role label. The input text is input into the trained strategy prediction model to predict the support strategy label. The two prediction labels are encoded and input into the grouped hybrid expert model to generate a preliminary response. The representative experience text most relevant to the input text is retrieved from the role experience library and filled into the preliminary response as supplementary prompt information. The final consultation response is generated and returned to the student client.

[0017] Based on the above solution, each step can be implemented in the following preferred specific manner.

[0018] As a preferred embodiment of the first aspect above, the specific process of step S1 is as follows: first, a preset number of student psychological counseling data of single-round question-and-answer dialogues are collected from the online psychological counseling website according to N psychological counseling topics and N counselor roles, and each counselor role corresponds to each psychological counseling topic one by one; then, the single-round question-and-answer dialogue text is converted into a multi-round dialogue question-and-answer text by prompting the large language model with preset prompt words, and the counselor role experience is inserted in batches into the multi-round dialogue text and the role-specific language style is embedded to generate dialogue text data containing the four-tuple of <student problem background information, counselor role label, support strategy label, context dialogue>, thereby forming a dialogue text dataset; wherein, each student's problem background information text is spliced ​​by the student's question, detailed description and topic keywords.

[0019] As a preferred embodiment of the first aspect mentioned above, in step S2, during the training process of the role matching model, the student's question background information text is encoded and used as input, and the role matching model outputs the predicted counselor role. The role matching model parameters are adjusted through the optimization algorithm to minimize the difference between the predicted counselor role and the counselor role label, and finally a trained role matching model is obtained.

[0020] As a preferred embodiment of the first aspect mentioned above, in step S3, during the training process of the strategy prediction model, after forming the problem background information text of each student, it is spliced ​​with the context dialogue text between the student and the counselor to obtain the response strategy text corresponding to the student, and then the response strategy text is encoded as input. The strategy prediction model outputs the predicted support strategy, and the strategy prediction model parameters are adjusted through the optimization algorithm to minimize the difference between the predicted support strategy and the support strategy label, and finally a trained strategy prediction model is obtained.

[0021] As a preferred embodiment of the above-mentioned first aspect, in step S4, the response strategy text is encoded using an embedding layer, and converted from a natural language representation to a vector representation of a unified dimension to form text input data, which serves as the input of the first Transformer block of the grouped hybrid expert model.

[0022] As a preferred embodiment of the above-mentioned first aspect, in step S4, the role expert group includes N role experts, and the strategy expert group includes M strategy experts. Each role expert or strategy expert corresponds to a pair of low-rank matrices, and each expert group corresponds to a gating network. The gating network corresponding to the role expert group is the first gating network, and the gating network corresponding to the strategy expert group is the second gating network.

[0023] As a preferred embodiment of the first aspect, the specific processing flow in the group hybrid expert layer described in step S4 is as follows:

[0024] S41: Encode all predefined consultant roles and all support strategies into corresponding low-dimensional vectors using a pre-trained language model to form a role embedding matrix and a strategy embedding matrix, respectively. Each row of the role embedding matrix corresponds to a role embedding vector, and each row of the strategy embedding matrix corresponds to a strategy embedding vector.

[0025] S42: Input the role embedding matrix into the first gating network, output the activation weight of each role expert, input the strategy embedding matrix into the second gating network, output the activation weight of each strategy expert;

[0026] S43: Perform low-rank decomposition on the parameter matrix that needs to be trained for each expert to obtain two low-rank matrices. The ratio of the scaling factor to the low-rank matrix dimension is used as the low-rank coefficient. The two low-rank matrices corresponding to each expert are multiplied by the low-rank coefficient to form a simulation parameter matrix corresponding to each expert.

[0027] S44: Multiplying the simulation parameter matrix of each role expert and the input hidden state as the output of each role expert, and weighting the outputs of all role experts based on the activation weight of the role expert to obtain the role representation output by the role expert group; multiplying the simulation parameter matrix of each strategy expert and the input hidden state as the output of each strategy expert, and weighting the outputs of all strategy experts based on the activation weight of the strategy expert to obtain the strategy representation output by the strategy expert group;

[0028] S45: Multiply the pre-trained parameter matrix by the input hidden state to obtain a weighted hidden state, and add the weighted hidden state, role representation, and strategy representation as the hidden state output by the grouped hybrid expert layer.

[0029] As a preferred embodiment of the above-mentioned first aspect, in the first gating network, each role embedding vector is multiplied by the gating parameter matrix of the first gating network and then normalized by the Softmax function to generate the activation weight of each role expert; in the second gating network, each strategy embedding vector is multiplied by the gating parameter matrix of the second gating network and then normalized by the Softmax function to generate the activation weight of each strategy expert.

[0030] As a preferred embodiment of the first aspect above, in step S6, the specific process of retrieving the representative experience text most relevant to the input text is: the input text is encoded by a large language model to form an emotion vector representation, the cosine similarity between the emotion vector representation and each emotion embedding vector in the character experience library is calculated, and the representative experience text with the highest cosine similarity is used as supplementary prompt information.

[0031] In a second aspect, the present invention provides a personalized student psychological counseling system based on a grouped hybrid expert model, which includes:

[0032] The data acquisition module is used to obtain student psychological counseling data from single-round question-and-answer dialogues. It uses a large language model to rewrite the single-round question-and-answer dialogue text into multi-round question-and-answer dialogue text according to preset prompt words, and embeds the counselor's role experience and language style to generate structured dialogue text data;

[0033] The first model training module is used to extract students' problem background information and counselor role labels from the conversation text data to form a role matching dataset, and train the role matching model on the role matching dataset;

[0034] The second model training module is used to extract support strategy labels, students' problem background information, and the context of the conversation between students and counselors from the conversation text data to form a strategy prediction dataset, and train the strategy prediction model on the strategy prediction dataset;

[0035] The third model training module is used to train a grouped mixture of experts model on the conversation text data. The grouped mixture of experts model is obtained by replacing the feedforward neural network layer in the Transformer block of the LLaMA model with a grouped mixture of experts layer. In the grouped mixture of experts layer, a gating network is used to route consultant role information to the role expert group and support strategy information to the strategy expert group. The two expert groups each learn specific semantic features, and the parameter scale is controlled by a low-rank method.

[0036] The role experience library construction module is used to obtain representative experience texts for each consultant role, and use the large language model to embed the emotion vectors of each representative experience text to generate the emotion embedding vectors. The emotion embedding vectors and the representative experience texts are combined into key-value pairs to form a role experience library for retrieval.

[0037] The consultation response generation module is used to use the student client to take the problem background information of the student who needs psychological counseling as input text, input the input text into the trained role matching model to predict the counselor role label, input the input text into the trained strategy prediction model to predict the support strategy label, encode the two prediction labels and input them into the grouped hybrid expert model to generate a preliminary response, retrieve the representative experience text most relevant to the input text from the role experience library and fill it into the preliminary response as supplementary prompt information, generate the final consultation response and return it to the student client.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] In response to the shortcomings of existing mental health counseling systems in terms of emotional support and empathy, the present invention proposes a personalized student psychological counseling method based on a grouped hybrid expert model, which is based on counselor role selection and empathy experience retrieval. The present invention dynamically selects a suitable counselor role and retrieves relevant empathy experiences based on the student's emotions and background to provide deep emotional resonance for the counseling process. This innovation enables the psychological counseling method of the present invention to adjust support strategies in real time to ensure effective help when facing different emotional needs. By introducing counselor roles and combining multi-dimensional emotional support strategies, the psychological counseling method of the present invention can provide users with stable, personalized and efficient emotional support under limited emotional data and role settings, expands its application scenarios in actual mental health services, and significantly improves the effectiveness of psychological counseling services and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flow chart of the steps of the method of the present invention;

[0041] Figure 2 A schematic diagram of the architecture of a Transformer block and a group hybrid expert layer provided in an embodiment of the present invention;

[0042] Figure 3 This is a system block diagram of the present invention. DETAILED DESCRIPTION

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more clearly understood, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in the various embodiments of the present invention can be combined accordingly without conflicting with each other.

[0044] In the description of the present invention, it should be understood that the terms "first" and "second" are used solely for descriptive purposes and are not to be construed as indicating or implying relative importance or implicitly specifying the number of technical features being described. Therefore, features defined as "first" or "second" may explicitly or implicitly include at least one of such features.

[0045] In order to solve the problem of insufficient emotional support and empathy in the existing technology, an embodiment of the present invention provides a personalized student psychological counseling method based on a grouped hybrid expert model. This method combines multi-dimensional empathy theory and dynamic adjustment of emotional support strategies, is trained based on psychological counseling data, and extracts features with emotional resonance from students' emotional changes and background information. Empathy experiences are inserted through experience retrieval, thereby enhancing the empathy ability of the psychological counseling method. In this way, the present invention can provide stable and effective emotional support in different situations, significantly improving the effectiveness of mental health services and user experience.

[0046] like Figure 1 As shown, in a preferred implementation of the present invention, the personalized student psychological consultation method based on the grouped hybrid expert model includes the following steps S1 to S6. The specific implementation process is described below.

[0047] S1: Obtain student psychological counseling data of single-round question-and-answer dialogues, use a large language model to rewrite the single-round question-and-answer dialogue text into multi-round question-and-answer dialogue text according to preset prompt words, and embed the counselor's role experience and language style to generate structured dialogue text data.

[0048] It should be noted that, in the present invention, the specific process of the above step S1 is as follows: first, a preset number of student psychological counseling data of single-round question-and-answer dialogues are collected from the online psychological counseling website according to N psychological counseling topics and N counselor roles, and each counselor role corresponds one-to-one to each psychological counseling topic; then, the single-round question-and-answer dialogue text is converted into a multi-round dialogue question-and-answer text by prompting the large language model (GPT-4o) with preset prompt words, and the counselor role experience is inserted in batches into the multi-round dialogue text and the role-specific language style is embedded to generate dialogue text data containing the four-tuple of <student problem background information, counselor role label, support strategy label, context dialogue>, thereby forming a dialogue text dataset.

[0049] It should be noted that, in this embodiment, the question of the mth student , detailed description and topic keywords Stitched together to form the student's question background information text , expressed as:

[0050]

[0051] Among them, ; represents the spliced ​​text; [Question] represents the student's question text content; [Desciption] represents the student's detailed description text content; [Keywords] represents the student's topic keyword text content.

[0052] In this embodiment, the conversation text dataset is divided into a training set and a test set in a ratio of 9:1 for subsequent model training and evaluation.

[0053] In this embodiment, the support strategy refers to dividing psychological counseling into three stages: exploration, comfort, and action. Different response strategies are used in each stage, such as asking questions, emotional response, retelling or interpretation, self-disclosure, encouragement and comfort, providing information, providing advice, and other eight strategies in total. These eight strategies are used in combination to complete the psychological counseling dialogue.

[0054] S2: Extract students’ problem background information and counselor role labels from the conversation text data to form a role matching dataset, and train the role matching model on the role matching dataset.

[0055] It should be noted that in step S2 of this embodiment, during the training process of the role matching model, the student's question background information text is encoded and used as input, and the role matching model outputs the predicted counselor role. The role matching model parameters are adjusted through the optimization algorithm to minimize the difference between the predicted counselor role and the counselor role label, and finally a trained role matching model is obtained.

[0056] It should be noted that in step S2 of this embodiment, the conversation text dataset in S1 is preprocessed to retain the student's personal background and the corresponding counselor role label, and the <student's problem background information, counselor role label> tuple is extracted to construct a role matching dataset. Then, based on the relevant experience and personality characteristics of each counselor role, a most suitable counselor role is manually assigned to each problem background information as the golden role label. During training, the student's problem background information text is encoded and input into the role matching model. The golden role label is used as the real label. The role matching model is trained to learn the connection between the student's problem background information and the golden role label. In the inference stage, the model can automatically recommend a suitable consultant role based on the student's problem background information. .

[0057] In the present invention, the role matching model can be selected by those skilled in the art according to actual needs. In this embodiment, the role matching model adopts the pre-trained language model RoBERTa (Robustly Optimized BERT Approach), the implementation of which belongs to the existing technology and will not be repeated here.

[0058] S3: Extract support strategy labels, students’ problem background information, and contextual conversations between students and counselors from the conversation text data to form a strategy prediction dataset, and train a strategy prediction model on the strategy prediction dataset.

[0059] It should be noted that in step S3 of this embodiment, during the training process of the strategy prediction model, after the background information text of the mth student's question is formed, it is compared with the contextual dialogue text between the student and the counselor. Splicing to get the corresponding response strategy text for the student , then encode the reply policy text as input, and the policy prediction model outputs the predicted support policy. The policy prediction model parameters are adjusted through the optimization algorithm to minimize the difference between the predicted support policy and the support policy label, and finally a trained policy prediction model is obtained. The above reply policy text can be expressed as:

[0060]

[0061] Among them, [Context] represents the contextual conversation text content between students and counselors.

[0062] It should be noted that in step S3 of this embodiment, after determining the counselor role, the text of the student's question, the counselor's reply text and their corresponding strategy labels are retained, and the <support strategy label, context dialogue> tuple is extracted to construct a strategy prediction dataset for training the strategy prediction model. Then, the most appropriate counseling strategy (support strategy) is manually assigned to the counseling scenario as the golden strategy label. During training, the response strategy text is encoded and input into the strategy prediction model, and the golden strategy label is used as the true label. The strategy prediction model is trained to learn the connection between the response strategy text and the golden strategy label. In the inference stage, the model can recommend a suitable support strategy based on the student's problem background information and the content of the conversation with the counselor. .

[0063] In the present invention, the strategy prediction model can be selected by those skilled in the art according to actual needs. In this embodiment, the strategy prediction model adopts the pre-trained language model RoBERTa (Robustly Optimized BERTApproach), the implementation of which belongs to the existing technology and will not be repeated here.

[0064] S4: A grouped mixture of experts model is trained on conversation text data. The grouped mixture of experts model is obtained by replacing the feed-forward neural network (FFN) layer in the Transformer block of the LLaMA (Large Language Model Meta AI) model with a grouped mixture of experts layer, while other parts of the Transformer block (such as the attention layer, residual connection, and normalization layer) remain unchanged. In the grouped mixture of experts layer, a gating network is used to route the consultant role information to the role expert group and the support strategy information to the strategy expert group. The two expert groups each learn specific semantic features and control the parameter scale through a low-rank method.

[0065] It should be noted that in step S4 of the present invention, an embedding layer is used to encode the reply strategy text, converting it from a natural language representation to a vector representation of a unified dimension to form text input data, which serves as the input of the first Transformer block of the grouped hybrid expert model.

[0066] It should be noted that in step S4 of the present invention, the role expert group includes N role experts, and the strategy expert group includes M strategy experts. Each role expert or strategy expert corresponds to a pair of low-rank matrices, and each expert group corresponds to a gating network. The gating network corresponding to the role expert group is the first gating network, and the gating network corresponding to the strategy expert group is the second gating network.

[0067] In step S4 of this embodiment, the internal structure of the group hybrid expert layer is as follows: Figure 2 As shown, its core design is to introduce two independent expert groups ( Figure 2 Group 1 and Figure 2 Group 2 in ), Group 1 corresponds to the Role Group, which is used to model various counselor roles in psychological counseling and includes N role experts, each of which processes the semantic features of the corresponding role; Group 2 corresponds to the Strategy Group: used to model support strategies and includes M strategy experts, each of which processes the semantic features of the corresponding strategy. Each expert corresponds to a pair of low-rank matrices ( Figure 2 in and , and … and , and ) to achieve efficient parameter representation and update. In addition, each expert group is equipped with an independent gating network to determine the degree of match between the input information and each expert, and dynamically select experts and assign activation weights based on the student input.

[0068] It should be noted that the specific processing flow in the group hybrid expert layer described in step S4 of the present invention is as follows:

[0069] S41: Encode all predefined consultant roles (N in total) and all support strategies (M in total) into corresponding low-dimensional vectors through a pre-trained language model (such as BERT), forming role embedding matrices respectively and the policy embedding matrix .

[0070] In this embodiment, each row of the role embedding matrix corresponds to a role embedding vector. The matrix consists of N role embedding vectors, each representing the global semantic representation of a role. Each row of the policy embedding matrix corresponds to a policy embedding vector. The matrix consists of M policy embedding vectors, each representing the global semantic representation of a policy. These embedding vectors are used in the gating network to activate the corresponding experts, thereby achieving dynamic modeling of the role-policy dimension.

[0071] Specifically, character embedding vector With the policy embedding vector It is expressed as follows:

[0072]

[0073] Where d represents the dimension of the role embedding vector or the policy embedding vector.

[0074] S42: Input the role embedding matrix into the first gating network, output the activation weight of each role expert, input the strategy embedding matrix into the second gating network, output the activation weight of each strategy expert.

[0075] In the first gating network, each role embedding vector is multiplied by the gating parameter matrix of the first gating network and then normalized by the Softmax function to generate the activation weight of each role expert; in the second gating network, each strategy embedding vector is multiplied by the gating parameter matrix of the second gating network and then normalized by the Softmax function to generate the activation weight of each strategy expert.

[0076] In step S42 of this embodiment, the input of the gated network is a role embedding vector or a policy embedding vector. The activation weight (i.e., contribution degree) of each expert is calculated through a set of learnable gated weight matrices. The higher the activation weight, the greater the contribution of the expert to the current input. The obtained activation weight is then processed by the Softmax function to ensure that the sum of all activation weights is 1, making the model selective.

[0077] No. character embedding vector The corresponding activation weight With the policy embedding vector The corresponding activation weight It can be expressed as:

[0078]

[0079] in, represents the gating parameter matrix of the first gating network; represents the gating parameter matrix of the first gating network.

[0080] S43: Perform low-rank decomposition on the parameter matrix that each expert needs to train to obtain two low-rank matrices. The ratio of the scaling factor to the low-rank matrix dimension is used as the low-rank coefficient. The two low-rank matrices corresponding to each expert are multiplied by the low-rank coefficient to form the simulation parameter matrix corresponding to each expert.

[0081] In step S43 of this embodiment, in order to achieve efficient parameter update and lightweight model, the LoRA (Low-Rank Adaptation) mechanism is introduced to perform low-rank decomposition on the parameter matrix of each expert. Here, the role expert and the policy expert perform the same operation, so no distinction is made. Taking the kth expert (which can be a role expert or a policy expert) as an example, this expert does not directly update the pre-trained parameter matrix , but use two low-rank matrices and To simulate parameter updates: , thus decomposing the high-dimensional embedding into multiple low-rank matrices for parameter constraints. Represents the simulation parameter matrix that needs to be trained; is the scaling factor, and r is the low-rank matrix dimension.

[0082] S44: Multiply the simulation parameter matrix of each role expert and the input hidden state x as the output of each role expert, and weight the output of all role experts based on the activation weight of the role expert to obtain the role representation output by the role expert group ; Multiply the simulation parameter matrix of each strategy expert and the input hidden state x as the output of each strategy expert, and weight the output of all strategy experts based on the activation weight of the strategy expert to obtain the strategy representation output by the strategy expert group :

[0083]

[0084]

[0085] In step S44 of this embodiment, the input hidden state x is the previous layer of the grouped mixed expert layer ( Figure 2 The output of the RMS normalization layer in .

[0086] S45: Multiply the pre-trained parameter matrix with the input hidden state to obtain the weighted hidden state. Add the weighted hidden state, role representation, and strategy representation as the hidden state output by the grouped hybrid expert layer. :

[0087]

[0088] S5: Obtain representative experience texts for each counselor role, and use a large language model to embed the sentiment vectors for each representative experience text to generate sentiment embedding vectors. The sentiment embedding vectors and representative experience texts are combined into key-value pairs to form a role experience library for retrieval.

[0089] It should be noted that step S5 of the present invention designs a role-based experience retrieval mechanism for building a role experience library. Specifically, first obtain the representative experience text corresponding to each consultant role. The representative experience text can be plot materials, script clips or film and television role content related to the consultant role identity background. The representative experience text of the consultant role is recorded as Then, each representative experience text is encoded into a corresponding sentiment embedding vector through a large language model. The representative experience text of the consultant role is recorded as , the emotional embedding vector and representative experience text belonging to the same character form a key-value pair , and finally form a role experience library for retrieval .

[0090] S6: The problem background information of the student who needs psychological counseling is used as input text through the student client. The input text is input into the trained role matching model to predict the counselor role label. The input text is input into the trained strategy prediction model to predict the support strategy label. The two prediction labels are encoded and input into the grouped hybrid expert model to generate a preliminary response. The representative experience text most relevant to the input text is retrieved from the role experience library and filled into the preliminary response as supplementary prompt information. The final consultation response is generated and returned to the student client.

[0091] It should be noted that in step S6 of the present invention, the specific process of retrieving the representative experience text most relevant to the input text is: the input text is encoded by a large language model to form an emotion vector representation, the cosine similarity between the emotion vector representation and each emotion embedding vector in the character experience library is calculated, and the representative experience text with the highest cosine similarity is used as supplementary prompt information.

[0092] In this embodiment, the retrieved representative experience text is added as supplementary prompt information to the preliminary response generated by the model, guiding the model to generate more empathetic and cognitive context-appropriate response content, thereby forming the final consultation response.

[0093] Next, the personalized student psychological counseling method based on the grouped hybrid expert model in the above embodiment is applied to a multi-role, multi-strategy dataset for evaluation index testing. The specific steps are as described in S1 to S6 and will not be repeated here. The main focus is on demonstrating the specific parameters and technical effects.

[0094] Example

[0095] In accordance with the implementation process of the aforementioned steps S1 to S6, this embodiment first collects real single-round question-and-answer psychological counseling data, and obtains a multi-role, multi-strategy psychological counseling data set through large language model preprocessing. Using these data as training data, randomly set seeds to train the model of the present invention and test its performance on the test set. During the evaluation process, the automation indicators (BLEU-1, ROUGE-L) in the field of natural language processing and the large language model evaluation indicators (Skillfulness, Empathy, Helpfulness) were used, and GPT-4o-2024-11-20 was used as the evaluation large language model. The comparison results of the method of the present invention with other models are shown in Table 1. The implementation methods of the MeChat model, SoulChat model, CPsyCounX model, and CharCounselor model in Table 1 belong to the prior art and will not be repeated here.

[0096] Table 1. Psychological counseling effectiveness evaluation form

[0097] This embodiment also demonstrates the visualization effect of the present invention in a real psychological consultation scenario, as shown below:

[0098] Student: I've been feeling really down lately. I can't sleep at night, and I'm constantly sleeping during the day. I wonder if it's because of the pressure of the high school entrance exam? Since last week, I've felt less and less inclined to work hard.

[0099] Counselor: [Question] I, Old Sun, see that you look so down. Come on, tell me, what's going on these days?

[0100] Student: I've had suicidal thoughts for a long time, but they've become more intense recently. I have no interest in anything, and sometimes I feel like crying and even trying to hurt myself.

[0101] Counselor: [Emotional response] Listening to your agony, I, Old Sun, feel like a mountain is pressing down on my heart, making it hard to breathe. It must be incredibly difficult for you, right? And your mind is filled with anxiety and helplessness, making it almost impossible to breathe. This torment is very similar to what I felt when I was trapped under the Five Elements Mountain.

[0102] Student: Yes, I think finishing the exam will be a relief. I'm really afraid of failure, afraid of disappointing my family, and even afraid of being called hypocritical if I hurt myself.

[0103] Counselor: [Retell or paraphrase] I, Old Sun, finally understand. Your stress probably comes from the combination of fear of losing and fear of what others will say. It's making you feel suffocated. Am I right?

[0104] Student: Yes, I often try to work hard, but then I feel like all my efforts are in vain. In short, nothing makes me happy. It's all I can do to avoid a complete breakdown.

[0105] Counselor: [Self-disclosure] Wow! You're holding up under such immense pressure. You're truly incredibly strong! It's just like when I faced Red Boy on my pilgrimage. That little demon nearly singed my hair with his flames. I wanted to give up, too! But in the end, I, Sun Wukong, gritted my teeth with perseverance and pushed through the challenge! You're doing something remarkable right now, too. Really, don't underestimate yourself! You're already very strong.

[0106] Student: Thank you for saying that, but I still can't control my crying. Every time I cry, it's like I'm trying to release something, but nothing changes afterward.

[0107] Counselor: [Offering advice] Crying is human nature, just like me swinging my club to beat the monster—it's just a burst of energy. But why not try running, or playing basketball a bit? Maybe moving around will help, like me swinging my club, dissipate the pressure inside!

[0108] Visiting student: Thank you for your suggestion. I didn't realize there were such specific, easy steps to try. I hope it really works.

[0109] Counselor: [Encouragement and reassurance] You're willing to take a step forward, which is already very strong! Take it one step at a time, and don't rush to fight yourself. You may find that every step you took was a great reward when you look back! If you have any questions, come to me, Old Sun!

[0110] The above text conversation demonstrates how the present method automatically matches the most appropriate counselor role (e.g., "Sun Wukong") based on the student's background information (primarily growth-related issues) and, combined with the conversation context, generates corresponding counseling strategy recommendations (indicated by square brackets). When the present method determines that the character's empathic experience needs to be introduced, it uses the emotion embedding vector to retrieve the most relevant representative experience text from the character's experience library and inserts it into the conversation in text form to enhance empathy. This demonstrates that the present method, through the synergistic mechanism of role matching, strategy prediction, and experience embedding, demonstrates excellent personalized emotional resonance capabilities in personalized psychological counseling tasks.

[0111] In summary, the role matching model of the present invention can match the corresponding counselor role according to the student's problem background information (such as age, gender, problem type, etc.). The matching of counselor roles and problem background information is based on the pre-definition of roles and problem types, such as "Sun Wukong" matches "growth" problems, "Juliet" matches "love" problems, etc. This matching relationship will determine which role experts the subsequent model will call to respond. The strategy prediction model can adopt psychological counseling strategies (such as empathetic listening, cognitive reconstruction, action guidance, etc.) based on the current student problem status and role setting. When a student needs psychological counseling, the strategy information will be passed into the grouped mixed expert model as a guidance signal, thereby generating a preliminary response with appropriate psychological intervention intentions. A grouped mixed expert layer is designed in the grouped mixed expert model, which adopts a grouped MoE (Mixture of Experts) structure. Two expert groups process role and strategy outputs, respectively, with different experts within each group representing different roles or strategies. A gating network dynamically regulates the outputs of multiple experts, acting as a "gate." Based on the user's current input context, it adaptively controls the activation level of each expert (e.g., role, strategy), enhancing the model's expressive and reasoning capabilities in different contexts and enabling personalized and diverse response generation. A role experience library is constructed, and by calculating the emotional similarity between the student's role experiences and those of multiple counselors in the library, the most resonant representative experience texts are retrieved as contextual cues, helping the model better understand the student's emotional state.

[0112] It should also be noted that the personalized student psychological counseling method based on the grouped mixed expert model in the above embodiment can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a personalized student psychological counseling system based on the grouped mixed expert model corresponding to the personalized student psychological counseling method based on the grouped mixed expert model provided in the above embodiment, such as Figure 3 As shown, it includes:

[0113] The data acquisition module is used to obtain student psychological counseling data from single-round question-and-answer dialogues. It uses a large language model to rewrite the single-round question-and-answer dialogue text into multi-round question-and-answer dialogue text according to preset prompt words, and embeds the counselor's role experience and language style to generate structured dialogue text data;

[0114] The first model training module is used to extract students' problem background information and counselor role labels from the conversation text data to form a role matching dataset, and train the role matching model on the role matching dataset;

[0115] The second model training module is used to extract support strategy labels, students' problem background information, and the context of the conversation between students and counselors from the conversation text data to form a strategy prediction dataset, and train the strategy prediction model on the strategy prediction dataset;

[0116] The third model training module is used to train a grouped mixture of experts model on the conversation text data. The grouped mixture of experts model is obtained by replacing the feedforward neural network layer in the Transformer block of the LLaMA model with a grouped mixture of experts layer. In the grouped mixture of experts layer, a gating network is used to route consultant role information to the role expert group and support strategy information to the strategy expert group. The two expert groups each learn specific semantic features, and the parameter scale is controlled by a low-rank method.

[0117] The role experience library construction module is used to obtain representative experience texts for each consultant role, and use the large language model to embed the emotion vectors of each representative experience text to generate the emotion embedding vectors. The emotion embedding vectors and the representative experience texts are combined into key-value pairs to form a role experience library for retrieval.

[0118] The consultation response generation module is used to use the student client to take the problem background information of the student who needs psychological counseling as input text, input the input text into the trained role matching model to predict the counselor role label, input the input text into the trained strategy prediction model to predict the support strategy label, encode the two prediction labels and input them into the grouped hybrid expert model to generate a preliminary response, retrieve the representative experience text most relevant to the input text from the role experience library and fill it into the preliminary response as supplementary prompt information, generate the final consultation response and return it to the student client.

[0119] It should also be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the various embodiments provided in this application, the division of steps or modules in the system and method is only a logical function division. In actual implementation, there may be other division methods, for example, multiple modules or steps can be combined or integrated together, and a module or step can also be split.

[0120] The embodiment described above is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Persons skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent substitution or equivalent transformation falls within the scope of protection of the present invention.

Claims

1. A personalized student psychological counseling method based on a grouped mixed expert model, characterized by: The following steps are involved: S1: Obtain student psychological counseling data of single-round question-and-answer dialogues, use a large language model to rewrite the single-round question-and-answer dialogue text into a multi-round question-and-answer dialogue text according to preset prompt words, and embed the counselor's role experience and language style to generate structured dialogue text data; S2: Extract students’ problem background information and counselor role labels from the conversation text data to form a role matching dataset, and train the role matching model on the role matching dataset; S3: Extract support strategy labels, students’ problem background information, and contextual conversations between students and counselors from the conversation text data to form a strategy prediction dataset. Train the strategy prediction model on the strategy prediction dataset. S4: A grouped mixture of experts model is trained on the conversational text data. The grouped mixture of experts model is obtained by replacing the feedforward neural network layer in the Transformer block of the LLaMA model with a grouped mixture of experts layer. In the grouped mixture of experts layer, a gating network is used to route the consultant role information to the role expert group and the support strategy information to the strategy expert group. The two expert groups each learn specific semantic features, and the parameter scale is controlled by a low-rank method. S5: Obtain representative experience texts for each consultant role, and use a large language model to embed sentiment vectors for each representative experience text to generate sentiment embedding vectors. The sentiment embedding vectors and representative experience texts are combined into key-value pairs to form a role experience library for retrieval. S6: The problem background information of the student who needs psychological counseling is used as input text through the student client. The input text is input into the trained role matching model to predict the counselor role label. The input text is input into the trained strategy prediction model to predict the support strategy label. The two prediction labels are encoded and input into the grouped hybrid expert model to generate a preliminary response. The representative experience text most relevant to the input text is retrieved from the role experience library and filled into the preliminary response as supplementary prompt information. The final consultation response is generated and returned to the student client.

2. The personalized student psychological counseling method based on the group hybrid expert model according to claim 1 is characterized in that: The specific process of step S1 is as follows: first, a preset number of student psychological counseling data of single-round question-and-answer dialogues are collected from an online psychological counseling website according to N psychological counseling topics and N counselor roles, where each counselor role corresponds to each psychological counseling topic; Subsequently, the large language model is prompted by preset prompt words to convert the single-round question-and-answer dialogue text into a multi-round dialogue question-and-answer text. The counselor role experience is inserted in batches into the multi-round dialogue text and the role-specific language style is embedded. The dialogue text data containing the four-tuple of <student problem background information, counselor role label, support strategy label, context dialogue> is generated, thus forming a dialogue text dataset; among them, the problem background information text of each student is spliced ​​together by the student's question, detailed description and topic keywords.

3. The personalized student psychological counseling method based on the group hybrid expert model according to claim 1 is characterized in that: In step S2, during the training process of the role matching model, the student's question background information text is encoded and used as input. The role matching model outputs the predicted counselor role. The role matching model parameters are adjusted through the optimization algorithm to minimize the difference between the predicted counselor role and the counselor role label, and finally a trained role matching model is obtained.

4. The personalized student psychological counseling method based on the group hybrid expert model according to claim 1 is characterized in that: In step S3, during the training process of the strategy prediction model, after forming the problem background information text of each student, it is spliced ​​with the contextual dialogue text between the student and the counselor to obtain the corresponding response strategy text of the student. The response strategy text is then encoded and used as input. The strategy prediction model outputs the predicted support strategy. The strategy prediction model parameters are adjusted through the optimization algorithm to minimize the difference between the predicted support strategy and the support strategy label, and finally a trained strategy prediction model is obtained.

5. The personalized student psychological counseling method based on the group hybrid expert model according to claim 4 is characterized in that: In step S4, the embedding layer is used to encode the reply policy text, converting it from natural language representation to a vector representation of uniform dimension to form text input data as the input of the first Transformer block of the grouped mixture of experts model.

6. The personalized student psychological counseling method based on the group hybrid expert model according to claim 1 is characterized in that: In step S4, the role expert group includes N role experts, and the strategy expert group includes M strategy experts. Each role expert or strategy expert corresponds to a pair of low-rank matrices. Each expert group corresponds to a gating network. The gating network corresponding to the role expert group is the first gating network, and the gating network corresponding to the strategy expert group is the second gating network.

7. The personalized student psychological counseling method based on the group hybrid expert model according to claim 6 is characterized in that: The specific processing flow in the group mixing expert layer described in step S4 is as follows: S41: Encode all predefined consultant roles and all support strategies into corresponding low-dimensional vectors using a pre-trained language model to form a role embedding matrix and a strategy embedding matrix, respectively. Each row of the role embedding matrix corresponds to a role embedding vector, and each row of the strategy embedding matrix corresponds to a strategy embedding vector. S42: Input the role embedding matrix into the first gating network, output the activation weight of each role expert, input the strategy embedding matrix into the second gating network, output the activation weight of each strategy expert; S43: Perform low-rank decomposition on the parameter matrix that needs to be trained for each expert to obtain two low-rank matrices. The ratio of the scaling factor to the low-rank matrix dimension is used as the low-rank coefficient. The two low-rank matrices corresponding to each expert are multiplied by the low-rank coefficient to form a simulation parameter matrix corresponding to each expert. S44: Multiplying the simulation parameter matrix of each role expert and the input hidden state as the output of each role expert, and weighting the outputs of all role experts based on the activation weight of the role expert to obtain the role representation output by the role expert group; multiplying the simulation parameter matrix of each strategy expert and the input hidden state as the output of each strategy expert, and weighting the outputs of all strategy experts based on the activation weight of the strategy expert to obtain the strategy representation output by the strategy expert group; S45: Multiply the pre-trained parameter matrix by the input hidden state to obtain a weighted hidden state, and add the weighted hidden state, role representation, and strategy representation as the hidden state output by the grouped hybrid expert layer.

8. The personalized student psychological counseling method based on the group hybrid expert model according to claim 7 is characterized in that: In the first gating network, each role embedding vector is multiplied by the gating parameter matrix of the first gating network and then normalized by the Softmax function to generate the activation weight of each role expert; in the second gating network, each strategy embedding vector is multiplied by the gating parameter matrix of the second gating network and then normalized by the Softmax function to generate the activation weight of each strategy expert.

9. The personalized student psychological counseling method based on the group hybrid expert model according to claim 1 is characterized in that: In step S6, the specific process of retrieving the representative experience text most relevant to the input text is as follows: the input text is encoded by a large language model to form an emotion vector representation, the cosine similarity between the emotion vector representation and each emotion embedding vector in the character experience library is calculated, and the representative experience text with the highest cosine similarity is used as supplementary prompt information.

10. A personalized student psychological counseling system based on a grouped hybrid expert model, characterized by: include: The data acquisition module is used to obtain student psychological counseling data from single-round question-and-answer dialogues. It uses a large language model to rewrite the single-round question-and-answer dialogue text into multi-round question-and-answer dialogue text according to preset prompt words, and embeds the counselor's role experience and language style to generate structured dialogue text data; The first model training module is used to extract students' problem background information and counselor role labels from the conversation text data to form a role matching dataset, and train the role matching model on the role matching dataset; The second model training module is used to extract support strategy labels, students' problem background information, and the context of the conversation between students and counselors from the conversation text data to form a strategy prediction dataset, and train the strategy prediction model on the strategy prediction dataset; The third model training module is used to train a grouped mixture of experts model on the conversation text data. The grouped mixture of experts model is obtained by replacing the feedforward neural network layer in the Transformer block of the LLaMA model with a grouped mixture of experts layer. In the grouped mixture of experts layer, a gating network is used to route consultant role information to the role expert group and support strategy information to the strategy expert group. The two expert groups each learn specific semantic features, and the parameter scale is controlled by a low-rank method. The role experience library construction module is used to obtain representative experience texts for each consultant role, and use the large language model to embed the emotion vectors of each representative experience text to generate the emotion embedding vectors. The emotion embedding vectors and the representative experience texts are combined into key-value pairs to form a role experience library for retrieval. The consultation response generation module is used to use the student client to take the problem background information of the student who needs psychological counseling as input text, input the input text into the trained role matching model to predict the counselor role label, input the input text into the trained strategy prediction model to predict the support strategy label, encode the two prediction labels and input them into the grouped hybrid expert model to generate a preliminary response, retrieve the representative experience text most relevant to the input text from the role experience library and fill it into the preliminary response as supplementary prompt information, generate the final consultation response and return it to the student client.

Images