IPT specificity and universality emotional psychological counseling robot system

Abstract

The invention discloses an IPT specificity and universality common-situation psychological counseling robot system, and the system comprises a database generation module which generates an IPT dialogue data record in combination with a Sourl Chat universality psychological counseling database according to a ChatGPT prompt project, obtains a depression outpatient dialogue record between a visitor role and a doctor role, and generates a common-situation IPT psychological counseling large database; the model construction module is used for constructing a special large model for the estrus-sharing IPT psychological counseling through LoRa fine tuning training based on a pre-training language model ChatGLM-6B according to the large database for the estrus-sharing IPT psychological counseling; and the interaction switching module is used for dynamically adjusting IPT-specific dialogues or universal emotional-sharing dialogues by adopting the emotional-sharing IPT-specific psychological counseling model according to dialogues, sentiment analysis and expression recognition of the consultant in a dialogue application scene between the consultant and the robot. The system can provide convenient, efficient and personalized psychological consultation service for the user, thereby being beneficial to relieving psychological pressure and improving psychological health level.

Description

Technical Field

[0001] The present invention relates to the fields of mental healthcare and artificial intelligence technology, and more specifically, to a dedicated and general-purpose empathy psychological counseling robot system for Interpersonal therapy (IPT). Background Art

[0002] Currently, with the increasing pressure in modern society, depression and anxiety have become common mental illnesses. Interpersonal therapy (IPT) is a short-term psychological therapy method centered on improving an individual's relationships with others. Traditional psychological therapies often require patients to go to medical institutions in person for treatment, but this method has many inconveniences, such as time and location restrictions, and it is difficult to meet the patients' needs for convenient, efficient, and personalized treatment. Therefore, the development of a psychological therapy robot software system that can provide convenient, efficient, and personalized treatment has become an urgent need. Summary of the Invention

[0003] In view of this, the present invention provides a dedicated and general-purpose empathy psychological counseling robot system for Interpersonal therapy. This system integrates artificial intelligence technology, psychological principles, and natural language processing technology, and can provide users with convenient, efficient, and personalized psychological counseling services, thereby effectively alleviating psychological pressure and improving mental health levels.

[0004] To achieve the above objectives, the present invention adopts the following technical solutions:

[0005] The present invention provides a dedicated and general-purpose empathy psychological counseling robot system for IPT, including:

[0006] A database generation module, used to combine the Soul Chat general-purpose psychological counseling database, generate IPT dialogue data records according to ChatGPT prompt engineering, obtain the depressive outpatient dialogue records between the client role and the doctor role, and generate a large empathy IPT psychological counseling database;

[0007] A model construction module, used to construct a dedicated large empathy IPT psychological counseling model based on the large empathy IPT psychological counseling database through LoRa fine-tuning training with the pre-trained language model ChatGLM-6B;

[0008] An interaction switching module, used to dynamically adjust the dedicated IPT dialogue or general-purpose empathy dialogue in the consultation scenario between the counselor and the robot according to the counselor's dialogue, sentiment analysis, and facial expression recognition, using the dedicated empathy IPT psychological counseling model.

[0009] Further, the database generation module includes:

[0010] An open-source dataset unit that performs data cleaning and structuring based on the open-source Soul Chat Chinese psychological counseling dataset;

[0011] A ChatGPT dialogue generation unit that generates dialogue texts conforming to the four major themes of IPT through ChatGPT prompt engineering techniques according to the rules of phased guidance, example-driven, and role division;

[0012] An acquisition unit for obtaining the depressive outpatient dialogue records between the client role and the doctor role;

[0013] A generation unit for generating an empathic IPT psychological counseling big database by combining the Soul Chat Chinese psychological counseling database, generating IPT dialogue texts according to ChatGPT prompt engineering, and obtaining depressive outpatient dialogue records.

[0014] Furthermore, the ChatGPT dialogue generation unit is specifically used to randomly generate empathic dialogue texts that conform to the interpersonal deficits, interpersonal conflicts, interpersonal grief, and role transition principles in IPT by adopting the prompt engineering guidelines of ChatGPT and the principles of interpersonal therapy psychology; associate and annotate the generated empathic dialogue texts with multi-modal data of videos, music, and images to form a multi-scenario enhanced dialogue template library.

[0015] Furthermore, the association annotation of the multi-modal data includes:

[0016] (a) Binding at least one video clip or image to each dialogue scenario to enhance the authenticity of the counseling situation;

[0017] (b) Classifying music clips through emotional labels and matching them with dialogue texts of corresponding emotional intensities.

[0018] Furthermore, in the model construction module, the pre-trained language model ChatGLM-6B includes:

[0019] Adopting a Transformer architecture with self-attention mechanism and feed-forward neural network layers, optimizing the decoder part, and adjusting the position of LayerNormalization;

[0020] Using bidirectional encoding in the understanding task to obtain information from the context on both the left and right sides simultaneously;

[0021] Using autoregressive decoding in the generation task to generate text in sequence and ensure semantic fluency and context consistency;

[0022] And adopting a relative position encoding mechanism to capture the relative relationships between sequences and improve the ability of long-context dependencies.

[0023] Furthermore, in the model construction module, fine-tuning training based on the pre-trained language model ChatGLM-6B through LoRa includes:

[0024] a) Introduce trainable low-rank matrices A and B in the attention layer of the ChatGLM-6B model, satisfying ΔW = AB T ;

[0025] b) Freeze the backbone network weights of the pre-trained model and only fine-tune the low-rank matrix parameters;

[0026] c) Adopt mixed-precision training and dynamic weight loading methods to reduce the video memory occupancy.

[0027] Furthermore, in the model construction module, fine-tuning training based on the pre-trained language model ChatGLM-6B through LoRa also includes:

[0028] d) Introduce reinforcement learning feedback during the training process to optimize the empathy and medical compliance of the generated content.

[0029] Furthermore, the interaction switching module includes:

[0030] A similarity calculation unit for calculating the cosine similarity between the visitor's answer and the pre-generated expected text to judge the semantic matching degree;

[0031] An analysis and recognition unit for respectively obtaining the text emotion score and the expression consistency score by combining the BERT emotion analysis model and the convolutional neural network expression recognition model;

[0032] A comprehensive switching value calculation unit for performing weighted calculation on the semantic matching degree, emotion score and expression score according to the preset weights to generate a comprehensive switching value S;

[0033] An interaction switching unit for switching from the IPT dedicated dialogue to the general empathy dialogue when S is lower than the threshold τ and returning to the IPT dialogue track after the preset conditions are met.

[0034] It can be seen from the above technical solutions that compared with the prior art, the present invention has the following technical advantages:

[0035] 1) Emphasize both data diversity and professionalism: Combine the Soul Chat general psychological counseling database, the IPT special dialogue data generated by ChatGPT, and the real depressive outpatient dialogue records to construct a multi-modal database covering a wide range of psychological scenarios, which not only ensures the generality of the data but also strengthens the professionalism for the core IPT issues such as interpersonal loss, conflict, sadness, and role transition.

[0036] 2) Model training with efficient domain adaptation: Based on the ChatGLM-6B pre-trained language model, fine-tuning is performed using the LoRa low-rank matrix factorization technique. Only a small number of parameter updates are required to achieve deep adaptation of the model to the psychological counseling scenario, significantly reducing the training cost while retaining the general language generation ability of the pre-trained model.

[0037] 3) Precise decision-making with dynamic interaction: Through a multi-modal fusion algorithm of cosine similarity, sentiment analysis, and facial expression recognition, the needs of the counselor are judged in real-time, and the IPT dedicated dialogue and general empathy dialogue modes are intelligently switched to ensure that the counseling process not only meets the treatment goals but also can flexibly respond to sudden emotional fluctuations, improving the counseling effect and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0039] Figure 1 It is a block diagram of the IPT dedicated and general empathy psychological counseling robot system provided by the present invention.

[0040] Figure 2 It is a schematic diagram of the principle of generating IPT dialogue data records using ChatGPT prompt engineering provided by the present invention.

[0041] Figure 3 It is a flowchart of LoRa fine-tuning training using the ChatGLM-6B large model provided by the present invention.

[0042] Figure 4 It is a schematic diagram of the IPT professional and general psychological counseling session interaction switching algorithm provided by the present invention.

[0043] Figure 5 It is a schematic diagram of an example of switching from an IPT professional dialogue to a general empathy IPT psychological counseling dialogue provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0045] Refer to Figure 1As shown, an embodiment of the present invention discloses an IPT-specific and universal empathy psychological counseling robot system, which has three interrelated core modules: a database generation module, which is used to generate an empathy IPT psychological counseling big database; a model construction module, which is used to create a large model dedicated to empathy IPT psychological counseling; an interactive switching module, which is used to realize session interactive switching according to the algorithm of IPT-specific and universal empathy psychological counseling session interactive switching.

[0046] Among them, the database generation module adopts the prompt engineering criteria of ChatGPT and the psychological principles of interpersonal therapy (IPT) to randomly generate empathic dialogue texts that conform to the principles of interpersonal loss, interpersonal conflict, interpersonal sadness, and role transformation in IPT, which are used for IPT professional consultation dialogues with visitors. The IPT professional consultation dialogue text also creatively integrates relevant videos, music, and images into the professional dialogues created by the ChatGPT prompt engineering. The open source Soul Chat Chinese psychological counseling database is used, and then the unique ChatGPT prompt engineering is integrated into the IPT professional dialogue library generated, and then combined with the real dialogue records between professional physicians and patients in the depression clinic, and finally an empathic IPT psychological counseling database is generated.

[0047] The Empathy IPT Psychological Counseling Special Model generated by the model building module builds a relaxed and open conversation environment, establishes a relationship of trust and understanding with users in a way of relaxing the body and mind, exchanging feelings or sharing experiences, and provides general psychological counseling help. The IPT professional counseling database based on the IPT psychology principle created by ChatGPT is combined with the Empathy IPT Psychological Counseling Special Model application, which is specifically used for psychological counseling dialogues that meet the interpersonal deficiencies, interpersonal conflicts, interpersonal sorrows, and role changes in psychological IPT. The creation of the Empathy IPT Psychological Counseling Special Model is based on the ChatGlm-6b Chinese language model open sourced by Tsinghua University, using the original Empathy IPT Psychological Counseling Big Database, and then successfully created through LoRa fine-tuning training.

[0048] The interactive switching module, based on the large model dedicated to empathetic IPT psychological counseling, realizes the interactive switching between IPT-specific psychological counseling and general psychological counseling sessions. Its algorithm part innovatively introduces conversation text similarity calculation, conversation text sentiment analysis algorithm, and dynamic visitor expression recognition results as the final decision on conversation interactive switching.

[0049] Finally, in specific implementation, for example, Audio2Face of NVIDIA's advanced metaverse system can be adopted to input voice into Audio2Face through the gRPC technology of computer software. In this way, the perfect combination of the robot's voice and the synthesis of the robot's lips and face can be realized, making the 3D robot speak vividly and realistically.

[0050] The above three modules will be described in detail as follows:

[0051] 1. Database generation module, which is used to combine the Soul Chat general psychological counseling database, generate IPT dialogue data records according to ChatGPT prompt engineering, obtain the depression outpatient dialogue records between the client role and the doctor role, and generate the empathy IPT psychological counseling big database.

[0052] Specifically, it includes:

[0053] Open-source dataset unit, which performs data cleaning and structuring processing based on the open-source Soul Chat Chinese psychological counseling dataset;

[0054] ChatGPT dialogue generation unit, which generates dialogue texts conforming to the four major themes of IPT through ChatGPT prompt engineering technology according to the rules of phased guidance, example-driven, and role division;

[0055] Obtaining unit, which is used to obtain the depression outpatient dialogue records between the client role and the doctor role;

[0056] Generation unit, which is used to combine the Soul Chat Chinese psychological counseling database, generate IPT dialogue texts according to ChatGPT prompt engineering, and obtain the depression outpatient dialogue records to generate the empathy IPT psychological counseling big database.

[0057] 1.1 Data source and processing

[0058] 1) General data integration: Extract dialogue records from the open-source Soul Chat Chinese psychological counseling dataset, clean the noise data (such as duplicate sentences, irrelevant symbols), and perform structured classification according to roles (client / counselor) and emotion labels (anxiety, depression, etc.).

[0059] 2) IPT special data generation: Based on ChatGPT prompt engineering, design a phased guidance template. For example:

[0060] Prompt example:

[0061] Role: Counselor (adhering to the IPT principle of interpersonal conflict resolution);

[0062] Goal: Guide the client to describe the conflict event and explore the differences in the needs of both parties.

[0063] Dialogue requirements: Include open-ended questions, emotional reflections, and suggestions for solutions.

[0064] By adjusting the prompt parameters, generate dialogue texts covering four major themes: interpersonal loss, conflict, sadness, and role transition. Each dialogue is associated with a video (such as a short film simulating a conflict scenario), music (such as soothing background music), or an image (such as a schematic diagram of role transition). Among them, the dialogue rules with ChatGPT are that the counselor is in the leading role, and the pre-generated answers of the client are used as the expected answers for the response text. Prompt Engineering is the design and optimization process for large language models (LLMs). By creating appropriate input prompts, it guides the model to generate high-quality outputs that meet the requirements.

[0065] Following the principles and specifications of ChatGPT Prompt Engineering, the following is an example of establishing a session database in JSON format using existing IPT session examples. Another example is to let ChatGPT create a video of "Beautiful Huangshan" for an IPT session theme related to interpersonal loss with Huangshan scenery.

[0066] Prompt

[0067] (1). Input file format

[0068] The input file contains multiple lines of dialogue, and each line is in a format similar to the following two cases:

[0069] Standard format: "Client: content" or "Counselor: content";

[0070] Non-standard format: Only the dialogue content, without a clear role prefix.

[0071] (2). Processing requirements

[0072] Each line of dialogue needs to be processed into JSON format, which includes the following fields:

[0073] Role: Indicate "Client" or "Counselor".

[0074] If a line does not have a clear role (non-standard format), then it defaults to inheriting the role of the previous line.

[0075] Content: The corresponding dialogue content.

[0076] Annotation: Split "content" into sentences, with each sentence as a separate annotation, and add an empty label field "label" to each sentence.

[0077] (3). Sentence division rules

[0078] Use "." as the sentence separator.

[0079] Each sentence ends with ".", ensuring a complete format.

[0080] (4). Output format

[0081] Each line of dialogue corresponds to a JSON object.

[0082] The output JSON file needs to contain the same number of lines as the input file, that is, JSON objects with the same number of lines.

[0083] (5). Output JSON example

[0084]

[0085]

[0086] (6). Solutions to fix non-standard formats

[0087] If a line lacks the prefix "Visitor:" or "Consultant:", inherit the role of the previous line.

[0088] Inheritance can only be done when the previous line has a clear role.

[0089] If the role cannot be determined (such as no prefix at the beginning), skip that line.

[0090] (7). Checking and verification

[0091] Ensure that the final output JSON file: data.json contains all JSON objects and has the same number of lines as the input file.

[0092] The theme of the generated "Beautiful Huangshan" video session is as follows:

[0093] Opening:

[0094] "Huangshan, a gem of China, is world-famous for its magnificent natural scenery! Here, there are four great wonders hidden, attracting countless tourists to come here.

[0095] Main body:

[0096] 1. Welcoming Pine

[0097] "On the mountainside of Huangshan, the Welcoming Pine is like a warm host, stretching out its long branches as if welcoming guests from afar. This pine tree has a history of over a thousand years and symbolizes friendship and enthusiasm.

[0098] 2. Bright Summit

[0099] "Reaching the highest point of Huangshan - Guangming Peak, you can enjoy a broad view with all the mountains of Huangshan in sight. At sunrise, with rays of light shining brightly, it seems as if you are in a fairyland on earth."

[0100] 3. Sea of Clouds

[0101] "One of the most fascinating wonders of Huangshan is the ever-changing sea of clouds. The peaks loom in the clouds and mists, just like a fairyland, extremely beautiful."

[0102] 4. West Sea Grand Canyon

[0103] "In the West Sea Grand Canyon, the steep valleys and secluded plank roads make people feel the shock and charm of nature."

[0104] End:

[0105] "The fresh green in spring, the coolness in summer, the red leaves in autumn, and the snow scenery in winter. Huangshan has unique charm in each season. What are you waiting for? Let's embark on a journey to Huangshan together!"

[0106] 3) Outpatient dialogue records: Based on the clinical criteria for diagnosing depression, initiate conversations with a real population including depression patients, and record the conversations of the visitors in the real population; record the depression outpatient dialogue records between the visitor role and the doctor role.

[0107] For the above three data sources, professional psychiatrists can be used to supervise them, and appropriate adjustments can be made according to the professional evaluation of the psychiatrists on the dialogue records in the above data.

[0108] 1.2 Multimodal data annotation

[0109] Video / image association: Use the FFmpeg tool to intercept video clips, bind them to the corresponding dialogue scenes (such as associating the "interpersonal conflict" dialogue with the video of the argument scene), and store the path and emotion tags through JSON annotation.

[0110] Music emotion matching: Extract music features (rhythm, pitch) based on the Librosa library, and classify them into types such as "sad", "calm", "inspiring" etc. in combination with an emotion dictionary (such as the NRC emotion intensity dictionary), and dynamically match them with the emotion intensity of the dialogue text (such as high anxiety, low depression).

[0111] In specific implementation, refer to Figure 2As shown, in the process of creating professional dialogues on interpersonal loss, interpersonal conflict, interpersonal sorrow, and role transition based on IPT using the principles and techniques of ChatGPT prompt engineering, through prompt engineering design, ChatGPT can generate counseling dialogues that conform to the IPT principle in a psychological context. The core lies in clear goals, phased guidance, example-driven, and tone control, ultimately generating coherent and psychologically logical content. This method can provide valuable support and simulation for the psychotherapy scenario.

[0112] Understand the medical background of IPT (Interpersonal Psychotherapy) - IPT is a short-term psychotherapy method centered on improving interpersonal relationships and is widely used in the intervention of mood disorders such as depression. Its core concepts include:

[0113] Focus areas: IPT focuses on the role of interpersonal problems in mental health, mainly covering four major themes:

[0114] 1. Grief / Loss: Deal with the emotional problems brought about by the loss of a loved one or an important relationship.

[0115] 2. Role Disputes: Deal with the troubles caused by role conflicts or inconsistent expectations.

[0116] 3. Role Transitions: Cope with the adaptation problems caused by major life changes (such as marriage, job leaving, etc.).

[0117] 4. Interpersonal Deficits: Improve social skills, deal with loneliness and other problems.

[0118] Goal-oriented: Alleviate emotional distress by improving interpersonal interaction patterns.

[0119] Explore the core role of prompt engineering:

[0120] When prompt engineering generates dialogues based on the IPT principle, it mainly affects the model through the following mechanisms:

[0121] 1. Define the task framework: Provide the background information and goals of IPT for the model to ensure that the dialogue conforms to medical principles.

[0122] 2. Define role division: Simulate real counseling scenarios through role assignment (such as "counselor" and "client").

[0123] 3. Phased guidance: Generate dialogues step by step according to the typical treatment steps of IPT (initial stage, treatment stage, end stage).

[0124] 4. Example-driven: Provide IPT-style dialogue examples to guide the model to imitate this style.

[0125] Clarify the goal and scenario: The task goal and dialogue scenario need to be clarified in the prompt. For example:

[0126] Goal: To help the client deal with the adaptation difficulties caused by role transitions.

[0127] Scenario: The first session between the counselor and the client, which requires building trust and initially identifying problems.

[0128] Example prompt: Please simulate a psychological counseling dialogue based on IPT principles, focusing on adaptation problems caused by role transitions (such as graduation, job change). The dialogue includes the statements of the counselor and the client. The counselor should demonstrate empathy, guide the exploration of problems, and help the client find initial coping methods.

[0129] Apply IPT principles to ChatGPT's dialogue generation, ensuring that the generated content focuses on the above areas and reflects psychological logic and treatment techniques.

[0130] Refine role and style requirements

[0131] The prompt needs to clarify the role division and style characteristics. For example:

[0132] Counselor: Has empathy, a gentle tone, and is good at guiding.

[0133] Client: Describes their own confusion and emotional experiences, and gradually opens up.

[0134] Example prompt:

[0135] Role:

[0136] - Counselor: Focus on empathy and guidance to help the client explore the root causes of problems.

[0137] - Client: Expresses uneasiness about life changes and expects to receive support.

[0138] The dialogue needs to meet the following characteristics:

[0139] - Each role speaks 2 - 4 sentences.

[0140] - The counselor's language reflects care, avoiding direct evaluation or strong suggestions.

[0141] Generate in stages

[0142] Design a step-by-step generation strategy according to the phased characteristics of IPT:

[0143] Initial stage: Build trust and confirm the treatment goal.

[0144] Mid - stage: Deeply explore the problem and explore feasible solutions.

[0145] End - stage: Consolidate the results and summarize the progress of the consultation.

[0146] Prompt design example:

[0147] Stage: Initial stage

[0148] Task: The counselor needs to guide the client to describe specific events and emotional reactions during the role transition to ensure that the other party feels understood.

[0149] Set up dialogue templates and examples

[0150] Strengthen the direction generated by the model through examples.

[0151] Example:

[0152] Client: I just changed my job recently, but I always feel unadapted and stressed.

[0153] Counselor: It sounds like the recent changes have made you feel a bit uneasy. Can you specifically talk about your feelings?

[0154] Embedding similar examples in the prompt can help the model better understand the characteristics of IPT - style conversations.

[0155] Details of the generation mechanism

[0156] Trigger the knowledge network of the model

[0157] ChatGPT learned texts related to psychology and medicine during the training process. Designing appropriate prompts can trigger these knowledge networks to generate content that conforms to the IPT concept.

[0158] Language control and tone optimization:

[0159] By clarifying tone requirements (such as empathy, supportive language), the prompt can control the tone and emotional characteristics of the output content.

[0160] Step - by - step guidance and logical chain:

[0161] The prompt breaks down complex tasks into specific steps, which helps to generate more logical and coherent conversations. For example:

[0162] First step: Confirm the problem area of the client.

[0163] Second step: Guide the expression of specific events and emotions.

[0164] Optimization strategy

[0165] Iteratively optimize step by step

[0166] Test the effects of different expressions through multiple rounds of adjusted prompts. For example:

[0167] Initial prompt: Lack of empathy in the generated conversation.

[0168] Improved prompt: Add requirements such as "show empathy" or "supportive language".

[0169] Add diverse examples

[0170] Provide the model with multiple IPT conversation examples to enhance its adaptability to different scenarios.

[0171] 2. The model construction module is used to construct a dedicated large model for empathy IPT psychological counseling based on the empathy IPT psychological counseling big database through LoRa fine-tuning training based on the pre-trained language model ChatGLM-6B.

[0172] The created dedicated large model for empathy IPT psychological counseling is mainly applicable to various groups and can provide general psychological counseling support for visitors. At the same time, it can also provide strong support for IPT professional psychological counseling. For example, when the visitor's answer does not meet the expected answer in the IPT professional conversation, a short-term general empathy psychological counseling support will be activated, and then it will return to the IPT professional psychological counseling conversation track. The dedicated large model for empathy IPT psychological counseling is based on the open-source ChatGlm-6b Chinese language large model of Tsinghua University, adopts the original empathy IPT psychological counseling big database, and then conducts LoRa fine-tuning training (Low-Rank Adaptation). LoRa is an efficient fine-tuning technology for large pre-trained language models (LLMs). Its core idea is to introduce a trainable low-rank matrix in the Transformer layer of the model to achieve model fine-tuning without changing the weights of the pre-trained model. This method can significantly reduce the number of training parameters, thereby reducing the demand for computing resources.

[0173] The ChatGLM-6B language large model is a pre-trained language model focusing on dialogue and generation tasks designed based on the GLM (General Language Model) architecture. Its structural design and optimization strategies have the following detailed features:

[0174] Optimization of the Transformer architecture

[0175] Core of the Transformer model: ChatGLM-6B is based on the standard Transformer architecture, with the core components of the self-attention mechanism and the feed-forward neural network layer, and can efficiently capture context information.

[0176] Decoder Structure: Different from traditional encoder-decoder architectures, ChatGLM-6B focuses more on optimizing the decoder part to adapt to generation tasks. Specifically, the decoder layer is improved, including: combining self-attention and convolution to enhance local dependencies. Position encoding is directly incorporated into the attention weights to reduce additional computations.

[0177] Position Adjustment of LayerNormalization: The Pre-Norm architecture (applying LayerNorm at the input layer) is adopted to accelerate model training and improve numerical stability.

[0178] Bidirectional and Autoregressive Modeling

[0179] Bidirectional Encoding: In understanding tasks, the model can obtain information from the context on both the left and right sides simultaneously, thus improving language understanding ability.

[0180] Autoregressive Decoding: In generation tasks, the model generates text sequentially to ensure semantic fluency and context consistency.

[0181] Relative Position Encoding

[0182] Relative Position Encoding Mechanism: Compared with absolute position encoding, using relative position encoding can better capture the relative relationships between sequences. This design is particularly suitable for processing long texts and enhances the ability to handle long context dependencies.

[0183] Hybrid Pre-training Objectives

[0184] ChatGLM-6B combines multiple task objectives during the pre-training stage:

[0185] Masked Language Model (MLM): By randomly masking parts of the input, it learns the bidirectional semantics of the language.

[0186] Causal Language Model (CLM): Optimizes text generation tasks in an autoregressive manner.

[0187] Fill-in-the-Blank Task: Supports generating and completing blank positions in the context, enhancing the ability to adapt to multiple tasks.

[0188] Chinese Optimization and Multilingual Capability

[0189] Pre-training on Chinese Corpus: The model has been deeply pre-trained on a large amount of Chinese data and is suitable for language understanding and generation tasks in Chinese scenarios.

[0190] Multi-language Support: Although centered on Chinese, ChatGLM-6B also supports multi-language tasks, especially in handling Chinese-English mixed contexts.

[0191] Dialogue Enhancement Capability

[0192] Multi-turn Dialogue Modeling: It supports tracking and generating context for multi-turn dialogue history to ensure logical and semantic consistency of dialogue content.

[0193] Human Preference Optimization: Optimize the model output through reinforcement learning (such as RLHF) to make it more in line with human dialogue needs and preferences.

[0194] Inference and Running Efficiency

[0195] ChatGLM-6B is optimized in the utilization of hardware resources and can run on a single GPU with 24GB video memory, which benefits from the following designs:

[0196] Parameter Sharing Mechanism: Reduce the model scale by sharing some network parameters.

[0197] Mixed Precision Calculation: Support FP16 and quantization acceleration technologies, significantly reducing video memory occupancy.

[0198] Dynamic Weight Loading: Load weights on demand during inference to improve memory efficiency.

[0199] In this embodiment, an efficient and lightweight fine-tuning technology based on LoRa (Low-Rank Adaptation) is adopted, which is designed specifically for low-cost customized fine-tuning training of the large model dedicated to empathy IPT psychological counseling. The application of LoRa on ChatGLM-6B reflects the following detailed features:

[0200] Core Idea: Low-rank Matrix Decomposition

[0201] In LoRa, instead of directly updating the original weights of the model, low-rank matrices A and B are introduced to approximate the change in weights: ΔW = A·B T

[0202] Where: A and B are low-rank matrices, usually with dimensions much smaller than the original weight matrix W. This decomposition method significantly reduces the number of parameters to be trained.

[0203] Improvement in Training Efficiency

[0204] Freeze the Trunk Network: Most of the weights of the model remain frozen during the fine-tuning process, and only the newly added parameter matrices of LoRa are trained. This method significantly reduces the computational resources required for training.

[0205] Video Memory Occupancy Optimization: The video memory occupancy of LoRa is much lower than that of full-parameter fine-tuning, making it suitable for consumer-grade GPU hardware (such as RTX3090 or A100).

[0206] Pluggable Module

[0207] Dynamic Loading: The fine-tuning parameters of LoRa are stored in independent modules, separated from the original model. These parameters can be loaded or removed as needed during inference to achieve dynamic switching between tasks.

[0208] Module Reusability: The fine-tuning parameters for different tasks can be stored independently and reused without retraining the entire model.

[0209] Combined Advantages with ChatGLM-6B

[0210] Dialogue Scene Customization: With LoRa, ChatGLM-6B can be quickly adapted to specific scenarios, such as professional fields like law, medicine, and education.

[0211] Enhanced Capability Local Fine-Tuning: For specific functions of ChatGLM-6B (such as the ability to understand long texts or knowledge supplementation in a certain field), only the relevant weights need to be adjusted.

[0212] Low Cost of Training Process

[0213] The significant reduction in parameters during LoRa fine-tuning means that:

[0214] Significantly Reduced Video Memory Requirements: Only a few GB of video memory is required to complete fine-tuning.

[0215] Shorter Training Time: Compared with full-parameter fine-tuning, the training speed is increased several times, and it can even be completed on ordinary hardware.

[0216] Scalability and Flexibility

[0217] Multi-Task Fine-Tuning: Supports fine-tuning on multiple different tasks simultaneously, with each task requiring only a small number of additional parameters.

[0218] High-Performance Guarantee: LoRa fine-tuning usually does not significantly reduce the original performance of the model, and can even achieve performance improvement through small-scale optimization.

[0219] Figure 3 Shows the principle process of training and generating a dedicated large model for Empathetic IPT psychological counseling by combining the ChatGLM-6B Chinese large model with the LoRa fine-tuning strategy. It includes the following steps: obtaining the ChatGLM-6B Chinese large model, verifying the performance of the basic model, preparing the Empathetic IPT psychological counseling database, data cleaning and enhancement, dividing the database, fine-tuning environment, defining the LoRa adapter, training the model, evaluating the Empathetic IPT large model, and optimizing model inference, and continuing to iterate.

[0220] Prepare the base model

[0221] 1. Obtain the chatBLM-6B model:

[0222] Obtain the weight file and configuration of the chatBLM-6B base model from the official or open-source community.

[0223] Ensure that the model is suitable for Chinese NLP tasks and supports further fine-tuning for the psychological counseling dialogue scenario.

[0224] 2. Verify the performance of the base model:

[0225] Test chatBLM-6B on Chinese basic tasks (such as text generation, classification, sentiment analysis, etc.) to ensure it has general language capabilities.

[0226] Data preparation: Empathetic IPT Psychological Counseling Big Database

[0227] 1. Organize psychological counseling data:

[0228] Ensure that the Empathetic IPT Psychological Counseling Database contains a large amount of high-quality dialogue data, covering the following:[[]]

[0229] Real conversations between counselors and clients.

[0230] Annotate empathetic behaviors, counseling strategies, and techniques (such as open-ended questions, emotional reflection, etc.) in the conversations.

[0231] Data format:

[0232] The data needs to be organized into a structured format (such as JSON, CSV) for subsequent fine-tuning.

[0233] 2. Data cleaning and augmentation:

[0234] Remove invalid conversations or noisy data to ensure the consistency of the training data.

[0235] Use data augmentation methods (such as synonym replacement, sentence pattern adjustment, etc.) to increase data diversity.

[0236] 3. Divide the dataset:

[0237] Divide the data into a training set, a validation set, and a test set, with a recommended ratio of 8:1:1.

[0238] The research adopts LoRa fine-tuning technology

[0239] 1. Understand the principle of LoRa fine-tuning:

[0240] LoRa (Low-Rank Adaptation) is an efficient fine-tuning technique that can fine-tune large models by adding a small number of trainable parameters, avoiding the high cost of training the entire model.

[0241] It is applicable to scenarios where GPU computing power is limited.

[0242] 2. Set up the fine-tuning environment:

[0243] Install necessary frameworks such as Hugging Face Transformers and LoRa libraries (such as peft or LoRa extensions).

[0244] Configure the hardware environment to ensure there is sufficient video memory support (more than 8GB of available video memory).

[0245] 3. Load the base model:

[0246] Load chatBLM-6B as the base model.

[0247] 4. Define the LoRa adapter:

[0248] Select the modules to be fine-tuned (such as the attention layer, Feed Forward layer).

[0249] Set the size of the low-rank matrix of the adapter (such as rank = 8) and the learning rate.

[0250] 5. Train the model:

[0251] Train the model using psychological counseling data. The specific steps include:

[0252] Define the loss function: Cross-entropy loss can be used for supervising dialogue generation.

[0253] Optimization strategy: Use the AdamW optimizer and set an appropriate learning rate (such as 2e-5).

[0254] During training, fine-tune the accuracy of dialogue generation and empathy behavior.

[0255] Evaluate the performance of the model on the validation set after each round of training.

[0256] Model evaluation

[0257] 1. Evaluation metrics:

[0258] BLEU / ROUGE: Evaluate the language quality of dialogue generation.

[0259] Accuracy: The accuracy of empathy behavior annotation.

[0260] Human evaluation: Invite professionals in psychology to conduct manual evaluation on the generated results to check the empathy ability and counseling effect.

[0261] 2. Improve the model:

[0262] According to the test set and the results of manual evaluation, adjust the training hyperparameters or further enhance the data.

[0263] Model optimization and deployment

[0264] 1. Optimize the inference efficiency:

[0265] Use quantization techniques (such as INT8 quantization) to optimize the inference speed of the model.

[0266] Deploy it to frameworks that support low-latency inference (such as ONNX, TensorRT).

[0267] 2. Integrate it into the application system:

[0268] Deploy the fine-tuned model to the psychological counseling platform to support real-time conversations and generate empathic responses.

[0269] The service can be provided in the form of API or web application.

[0270] 3. Continuous iteration:

[0271] Collect user feedback, continuously update the database and fine-tune the model to ensure the gradual optimization of the counseling effect.

[0272] 3. The interaction switching module is used to dynamically adjust the IPT-specific dialogue or the general empathic dialogue according to the conversation, sentiment analysis, and expression recognition of the counselor in the scenario of the counselor's conversation with the robot, using the empathic IPT-specific psychological counseling model. Specifically, it includes:

[0273] The similarity calculation unit is used to calculate the cosine similarity between the visitor's answer and the pre-generated expected text to judge the semantic matching degree;

[0274] The analysis and recognition unit is used to respectively obtain the text sentiment score and the expression consistency score by combining the BERT sentiment analysis model and the convolutional neural network expression recognition model;

[0275] The comprehensive switching value calculation unit is used to perform weighted calculation on the semantic matching degree, sentiment score, and expression score according to the preset weights to generate the comprehensive switching value S;

[0276] The interaction switching unit is used to switch from the IPT-specific dialogue to the general empathic dialogue when S is lower than the threshold τ, and return to the IPT dialogue track after the preset conditions are met.

[0277] The algorithm for switching between the specificity and generality of IPT empathy psychological counseling conversations is mainly applied in professional IPT psychological counseling. For the off-topic answers of the client, an algorithm is needed to decide whether to initiate a short-term empathy IPT general psychological counseling conversation. The cosine similarity Ai model measures the similarity between two text vectors by calculating the cosine value of the angle between them. In machine learning research, cosine similarity is often applied to the learned low-dimensional feature embeddings through conversation texts to quantify the semantic similarity between high-dimensional objects. Although the algorithm is effective in most cases, its performance deteriorates when the angle between the two vectors approaches more than 70 degrees. Therefore, the algorithm for interactive switching introduces the session text sentiment analysis algorithm and the dynamic client facial expression recognition results as additional steps to finally decide whether to initiate the session interactive switching.

[0278] Here, the client's answer and the text of the answer expected by the robot are two vectors a and b. When the angle between the text vectors a and b exceeds 70 degrees, it indicates that the text similarity is basically dissimilar. Then, it is necessary to enter the session text sentiment analysis algorithm link. The following is the process of sentiment analysis adopted:

[0279] Sentiment representation

[0280] Polarity classification: Determine whether the text sentiment is positive, negative, or neutral.

[0281] Sentiment intensity: Evaluate the strength of the sentiment.

[0282] Multi-dimensional sentiment model: For example, adopt the sentiment space model (such as the sentiment wheel) to represent the complexity of sentiment (such as anger, joy, fear).

[0283] Feature extraction

[0284] Lexical features: Extract sentiment-related words based on the sentiment dictionary.

[0285] Syntactic features: Analyze the syntactic structure and dependency relationship of the text.

[0286] Context features: Consider the context semantics of the text to avoid misjudgment (such as puns, irony).

[0287] Based on the deep learning method, the BERT language model is mainly adopted:

[0288] Neural network model:

[0289] Common models: Recurrent neural network (RNN), Long short-term memory network (LSTM), Convolutional neural network (CNN).

[0290] Features: Capture semantics through distributed representation learning (word embeddings, such as Word2Vec, GloVe).

[0291] Pre-trained language models:

[0292] For example: BERT, GPT, RoBERTa. Through pre-training on a large-scale corpus, they can capture context information, adapt to various sentiment analysis tasks, and perform well in complex situations.

[0293] Finally, by collecting the real-time dynamic facial expressions of the visitor, facial expression recognition is trained using a deep convolutional neural network: The important pixel information of the previous layer is filtered and placed in the next layer (Feature layer). For specific details, existing facial expression recognition technologies on the market can be referred to. This can effectively simplify a large amount of raw pixel information, thereby improving the image processing efficiency.

[0294] Such as Figure 4 As shown, the algorithm for the psychological counseling session interaction switching value S:

[0295] S = w t ·S t + w e ·S e + w c ·S c

[0296] Among them, S t represents the text sentiment analysis score of the visitor, in the range [0, 1], indicating the consistency or expected matching degree of the sentiment.

[0297] S e represents the facial expression analysis score of the visitor, in the range [0, 1], indicating the consistency between the facial expression and the expected emotional expression.

[0298] S c represents the matching degree score between the text of the visitor's answer and the pre-generated expected text, in the range [0, 1], indicating the consistency of the language content.

[0299] w t ,w e ,w c : respectively represent the weights of text, expression, and content matching, satisfying w t + w e + w c = 1.

[0300] Judge whether it meets the expectation: If S ≥ τ, it meets; if S < τ, it does not meet. τ is used as a threshold, in the range [0, 1], and can be adjusted according to the specific scenario. For example, τ = 0.7.

[0301] If the S - value calculation meets the expected value of around 0.7, it is decided that an interactive switch is needed between IPT professional psychological counseling conversations and general empathy psychological counseling conversations, that is, briefly switch from IPT professional conversations to general empathy IPT psychological counseling conversations.

[0302] For example:

[0303] Refer to Figure 5 As shown, for the example of switching from IPT professional conversations to general empathy IPT psychological counseling conversations, when the IPT interpersonal conflict professional conversation reaches the sixth line, the answer from the visitor obtained by the robot system does not match the expected answer. Then, through the conversation interaction switching algorithm, it is decided to briefly start the "general empathy IPT psychological counseling conversation". When it reaches the 22nd line, it returns to the track of the IPT interpersonal conflict professional conversation.

[0304] The IPT - specific and general - empathy psychological counseling robot system provided by the present invention can further integrate the NVIDIA Audio2Face engine, and through gRPC technology, synchronize voice input with 3D virtual face animation in real - time, realizing the matching of the robot's expression, lip - shape and voice. It can be deployed on a cloud server and supports data interaction with mobile terminals and VR devices through API interfaces. By integrating AI technology and psychological principles, this system can build a psychological counseling platform with professionalism, flexibility and low cost, which helps to provide more convenient and personalized treatment support for patients with depression, anxiety disorders, etc.

[0305] In this specification, each embodiment is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0306] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. An IPT-specific and universal empathy psychological counseling robot system, characterized in that: include: The database generation module is used to combine the Soul Chat universal psychological counseling database and generate IPT dialogue data records based on the ChatGPT prompt project, and obtain the depression clinic dialogue records between the visitor role and the doctor role to generate an empathy IPT psychological counseling database; A model building module is used to build a special large model for empathy IPT psychological counseling based on the empathy IPT psychological counseling big database and the pre-trained language model ChatGLM-6B through LoRa fine-tuning training; The interactive switching module is used to dynamically adjust the IPT-specific dialogue or the universal empathy dialogue according to the dialogue, emotion analysis, and expression recognition of the consultant in the application scenario of the dialogue between the consultant and the robot, using the empathy IPT-specific psychological counseling model.

2. According to claim 1, an IPT-specific and universal empathy psychological counseling robot system is characterized in that: The database generation module comprises: The open source data set unit performs data cleaning and structural processing based on the open source Soul Chat Chinese psychological counseling data set; The ChatGPT dialogue generation unit generates dialogue texts that conform to the four major themes of IPT through ChatGPT prompt engineering technology, following the rules of phased guidance, example drive, and role division; An acquisition unit, used for acquiring the depression clinic dialogue record between the visitor role and the doctor role; The generation unit is used to combine the Soul Chat Chinese psychological counseling database, generate IPT dialogue text according to the ChatGPT prompt project, and obtain depression clinic dialogue records to generate an empathy IPT psychological counseling database.

3. The IPT-specific and universal empathy psychological counseling robot system according to claim 2 is characterized in that: The ChatGPT dialogue generation unit is specifically used to adopt the prompt engineering criteria and interpersonal therapy psychology principles of ChatGPT to randomly generate empathy dialogue texts that conform to the principles of interpersonal deficiency, interpersonal conflict, interpersonal sadness, and role transformation in IPT; and associate and annotate the generated empathy dialogue texts with multimodal data of videos, music, and images to form a multi-scenario enhanced dialogue template library.

4. The IPT-specific and universal empathy counseling robot system according to claim 3 is characterized in that: The multimodal data is associated and annotated, including: (a) Bind at least one video clip or image to each dialogue scene to enhance the authenticity of the consultation situation; (b) Music clips are classified by emotion tags and matched with dialogue texts of corresponding emotion intensity.

5. The IPT-specific and universal empathy psychological counseling robot system according to claim 1 is characterized in that: In the model building module, the pre-trained language model ChatGLM-6B includes: The Transformer architecture with self-attention mechanism and feed-forward neural network layer is adopted, the decoder part is optimized, and the LayerNormalization position is adjusted; Bidirectional encoding is used in comprehension tasks, obtaining information from both the left and right contexts simultaneously; Autoregressive decoding is used in the generation task to generate text in sequence and ensure semantic fluency and context consistency; A relative position encoding mechanism is used to capture the relative relationship between sequences and improve the ability of long context dependencies.

6. The IPT-specific and universal empathy psychological counseling robot system according to claim 1 is characterized in that: In the model building module, the pre-trained language model ChatGLM-6B is fine-tuned through LoRa, including: a) Introduce trainable low-rank matrices A and B in the attention layer of the ChatGLM-6B model to satisfy ΔW = AB T ; b) Freeze the backbone network weights of the pre-trained model and only fine-tune the low-rank matrix parameters; c) Use mixed precision training and dynamic weight loading to reduce video memory usage.

7. The IPT-specific and universal empathy psychological counseling robot system according to claim 6 is characterized in that: The model building module is based on the pre-trained language model ChatGLM-6B and is fine-tuned through LoRa, and also includes: d) Introduce reinforcement learning feedback during the training process to optimize the empathy and medical compliance of the generated content.

8. The IPT-specific and universal empathy counseling robot system according to claim 1 is characterized in that: The interactive switching module includes: A similarity calculation unit is used to calculate the cosine similarity between the visitor's answer and the pre-generated expected text to determine the semantic matching degree; The analysis and recognition unit is used to combine the BERT sentiment analysis model and the convolutional neural network expression recognition model to obtain the text sentiment score and expression consistency score respectively; A comprehensive switching value calculation unit, used to perform weighted calculation on the semantic matching degree, the emotion score and the expression score according to preset weights to generate a comprehensive switching value S; The interactive switching unit is used to switch from the IPT-specific dialogue to the general empathy dialogue when S is lower than the threshold τ, and return to the IPT dialogue track after the preset conditions are met.

Images