Psychological model fine-tuning method and system based on five-dimensional role and closed-loop feedback

Abstract

This invention discloses a method and system for fine-tuning a psychological model based on five-dimensional roles and closed-loop feedback. The method generates role-playing training data by constructing a five-dimensional structured role profile system and combines it with a data replay mechanism to build a hybrid training set, enabling the model to acquire a specific role expression style while retaining its psychological professional capabilities. This invention employs a dual-track scoring and closed-loop feedback optimization strategy to quantify the scoring of the role-playing and psychological counseling dimensions, synergistically enhancing role-playing ability, suppressing catastrophic forgetting, and dynamically improving the intensity of professional replay. This invention eliminates the need for crowdsourcing to construct high-quality role-playing data and, through the closed-loop feedback mechanism, significantly improves the model's conversational affinity and assessment objectivity in real-life practical scenarios while maintaining a professional psychological core.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence and natural language processing technology, and further relates to a method and system for fine-tuning a large-scale mental health model based on five-dimensional role modeling and data replay closed-loop feedback in the fields of instruction tuning and domain adaptation of large language models (LLM). This invention can be used to construct a highly human-like and professional intelligent psychological counseling dialogue model, providing users with personalized psychological counseling services through intelligent technology. Background Technology

[0002] Existing psychological counseling models are trained through tasks such as psychological intention recognition, causal reasoning, and psychological guidance. The Mental-LLM paper points out that fine-tuning instructions can make small-to-medium-sized LLMs comparable to trillion-level models in mental health prediction, but strict ethical regulations are needed for their use. EmoLLM significantly improves multimodal emotion understanding through multi-perspective visual modeling and emotional cueing technology. SoulChat resolves the contradiction between user aversion to one-turn dialogue and the weakening of model suggestion capabilities in multi-turn dialogue. Large-scale models related to psychological counseling have been sufficiently developed, each with its own advantages and disadvantages. A common contradiction exists in existing technologies: the "professionalism" and "humanity" of a model are often negatively correlated. Training with professional data makes the model rigid and lacking in empathy; training with everyday, role-playing data dilutes its professional capabilities.

[0003] iFlytek Co., Ltd. discloses a training method, device, electronic device, and storage medium for a psychological counseling model in its patent application "Training Method, Device, Electronic Equipment, and Storage Medium for a Psychological Counseling Model" (Application No. 202411955110.3, Publication No. CN 119742030 A). The method includes: acquiring sample psychological counseling information for multiple sample clients corresponding to a target psychological scenario category; determining the age of the sample clients and the sample psychological guidance strategy corresponding to the target psychological scenario category; inputting the sample psychological counseling information and sample psychological guidance strategy into a large language model to obtain sample counseling plans for different counseling stages; and training the initial psychological counseling model based on the sample counseling plans of each sample client to obtain the psychological counseling model. This method trains the psychological counseling model based on sample counseling plans for different counseling stages of each sample client. Therefore, it can generate counseling plans for different counseling stages based on this psychological counseling model, thereby improving the stylistic diversity of psychological counseling dialogues obtained based on counseling plans for different counseling stages. However, this method still has shortcomings. While it primarily improves recognition accuracy through multi-task joint training and uncovers real user problems, it fundamentally relies on traditional training paradigms and lacks targeted optimization for "role-playing flexibility" and "low-cost data construction." Specifically, this method fails to resolve the contradiction between professional consulting language and a natural, approachable style, resulting in dialogues that are often overly serious and lack warmth. Furthermore, its training data acquisition costs are high, making it difficult to quickly adapt to changing personalized role requirements. In addition, the method lacks an objective, quantitative, automated evaluation mechanism, making it difficult to avoid subjective biases from human evaluation.

[0004] Ping An Technology (Shenzhen) Co., Ltd. disclosed an AI-based psychological counseling method in its patent application "AI-based Psychological Counseling Method, Device, Equipment and Storage Medium" (Application No. 202411447258.6, Publication No. CN 119339891 A), aiming to provide personalized psychological counseling services through intelligent technology. This method first acquires the user's voice data and converts it into text data using automatic speech recognition technology. Then, it analyzes the user's voice and text data to extract user characteristic information, including gender, age, and emotional characteristics. Based on these analysis results, it generates prompt word templates to guide the target psychological counseling model in generating personalized text-based psychological counseling suggestions that meet the user's needs. This method significantly improves the real-time nature and relevance of psychological counseling through automated and intelligent analysis processes, providing users with more precise mental health support. However, a remaining shortcoming is that this method mainly relies on front-end speech recognition and feature extraction technologies and prompt word engineering, rather than deeply fine-tuning the underlying parameters of the model. This makes the model less stable when handling long-context, multi-turn dialogues, and once an error occurs in speech recognition, subsequent processes will produce a chain of errors. More importantly, the method lacks specific design for "role-playing" scenarios, making it difficult to flexibly simulate the language styles of different social roles while maintaining psychological professionalism, resulting in a disjointed user experience.

[0005] In summary, existing large-scale psychological counseling models suffer from the following drawbacks: First, the models lack approachability because their training data is overly specialized, resulting in a lack of natural, conversational expression. Second, high-quality role-playing data relies heavily on crowdsourced human resources, which is costly and difficult to scale. Third, existing role-playing methods lack flexibility and are difficult to adapt quickly to different roles and application scenarios. Fourth, existing technologies focus primarily on theory while neglecting practical, everyday scenarios, leading to a disconnect from actual user needs. Fifth, mainstream assessment schemes rely on subjective human evaluation, resulting in insufficient objectivity. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of the existing technologies by providing a method and system for fine-tuning a psychological model based on five-dimensional role modeling and closed-loop feedback. This aims to solve the problems of high data construction costs, difficulty in balancing professionalism and accessibility, and lack of objective evaluation closed loops in the existing technologies.

[0007] The technical approach to achieving the invention's objective is to address the challenges of data acquisition difficulties, easy forgetting during model training, and high evaluation costs in psychological counseling scenarios by proposing a complete data-model closed-loop optimization scheme. Specifically, firstly, to address the shortage of high-quality role-playing data, this invention utilizes the strong generative capabilities of a large language model combined with five-dimensional role profiles to automatically generate high-quality synthetic data covering a wide range of life scenarios in batches, fundamentally solving the data scarcity bottleneck. Secondly, to address the catastrophic forgetting problem during fine-tuning, this invention introduces a data replay mechanism and constructs a hybrid training set to ensure a balance between "humanization" and "professionalism" in the model. Thirdly, to address the pain points of difficulty in monitoring data generation quality and high manpower costs for manual evaluation, this invention constructs a dual-track automated evaluation system based on a large model. This system uses a large-parameter, large-model approach to objectively quantify scores from both the "role-playing dimension" and the "psychological counseling dimension," and automatically triggers a closed-loop feedback strategy based on the scoring results. This ensures the quality of generated data while significantly reducing manpower costs, achieving fully automated closed-loop iteration throughout the entire process.

[0008] To achieve the above objectives, the specific implementation steps of the present invention include the following:

[0009] Step 1: Generate role-playing training data based on the constructed five-dimensional structured character profile system;

[0010] Step 2: Randomly select a portion of the basic psychological counseling dataset according to a specific ratio, and replay the role-playing training data to obtain a mixed training set;

[0011] Step 3: Use the basic psychological counseling dataset to fine-tune the base model using LoRA to obtain the fine-tuned model. ;

[0012] Step 4: Use a mixed training set to fine-tune the model. LoRA was fine-tuned by blending character styles to obtain the adjusted model. ;

[0013] Step 5: Use a large model with many parameters to fine-tune the model. The batch output results are scored using a dual-track system, based on both the "role-playing dimension" and the "psychological counseling dimension".

[0014] Step 6: If the score for the role-playing dimension is lower than its preset threshold, increase the sample size of the role-playing training data in the mixed training set to generate a new mixed training set, and then repeat Step 4; if the score for any item in the psychological counseling dimension is lower than its preset threshold, increase the data replay ratio in the mixed training set. After generating a new mixed training set, repeat step 4;

[0015] Step 7: Determine whether the individual dimension score and the total score have both reached the corresponding threshold or the preset maximum number of feedback rounds. If so, the trained mental model is obtained; otherwise, proceed to step 5.

[0016] Furthermore, the structured character profile system built on five dimensions includes basic background, emotional characteristics, interaction patterns, relationship networks, and psychological support dimensions. The basic background includes basic information such as the age, gender, occupation, and educational background of the virtual character to be portrayed. Emotional characteristics refer to the personality tone and common emotional response patterns of the virtual character. Interaction patterns refer to the language style, speaking speed, and commonly used catchphrases of the virtual character. The relationship network refers to the strength of the relationship and interaction boundaries between the virtual character and the user. The psychological support dimension refers to the strategic preferences of the virtual character when providing psychological support. The virtual character to be portrayed is a virtual personality with specific social identity, psychological traits, and language habits, concretely simulated according to the needs of the consultation scenario through the parameterized definition of the above five dimensions; it is not a specific natural person.

[0017] Furthermore, the role-playing training data refers to a JSON-formatted dataset that includes five-dimensional character profiles, dialogue topics, task instructions, and prompts for enforcement rules, which drives the large language model to generate the dataset.

[0018] Furthermore, the step of randomly selecting a portion of the basic psychological counseling dataset at a specific ratio and replaying the role-playing training data is as follows:

[0019] The first step is to calculate the amount of replay data. : ,in, The initial data replay ratio is 0.01. This represents the total amount of data in the basic psychological counseling dataset;

[0020] The second step involves randomly selecting samples from the basic psychological counseling dataset. The data is shuffled and mixed with the role-playing training data generated in step 1 to obtain a mixed training set for replaying the role-playing training data.

[0021] Furthermore, the steps for fine-tuning the base model using the basic psychological counseling dataset using LoRA are as follows:

[0022] The first step is to use the open-source large language model as the model to be fine-tuned in this step;

[0023] The second step involves configuring the LoRA adapter parameters and training the model using a publicly available mental health instruction fine-tuning dataset at a preset learning rate to obtain a fine-tuned model with a professional psychological core. .

[0024] Furthermore, the fine-tuned model is trained using a mixed training set. The steps for fine-tuning LoRA's character style are as follows:

[0025] The first step is to fine-tune the model. This serves as the model to be fine-tuned in this step;

[0026] The second step involves using a fine-tuning model. At the same learning rate, while preserving the model's psychological expertise, LoRA fine-tunes the model to obtain a specific character expression style, resulting in a fine-tuned model. .

[0027] Furthermore, the steps for using a large model with many parameters to score the "role-playing dimension" and "psychological counseling dimension" on a dual-track basis are as follows:

[0028] The first step is to fine-tune the model for the current round using a large number of parameters and a large model. The batch output results were scored based on three aspects: consistency, anthropomorphism, and attractiveness, to determine the "role-playing dimension." , The range of values ​​is ;

[0029] The second step is to fine-tune the model for the current round using a large model with many parameters. The batch output results score the "psychological counseling dimension" from three aspects: emotional empathy, cognitive empathy, and guidance ability. , The range of values ​​is ;

[0030] The third step is to apply the preset weighting coefficients. and Calculate the total score ,in, and The range of values ​​is .

[0031] Furthermore, the fact that the role-playing dimension score is below its preset threshold means that the preset threshold is 90% of the upper limit of the model score. When the values ​​fall below a threshold, the model is deemed to have insufficient role consistency, anthropomorphism, or attractiveness. In this case, the system automatically triggers feedback, adding 300 newly generated role-playing training data points to the mixed training set to form a new mixed training set, thereby enhancing the role-playing ability.

[0032] Furthermore, the statement that a score for a single item in the psychological counseling dimension is lower than its preset threshold means that the preset threshold is 90% of the upper limit of the model's score. When the data falls below a certain threshold, the system determines that the model has experienced catastrophic amnesia, leading to a decline in emotional empathy, cognitive empathy, and guidance capabilities. At this point, the system automatically triggers feedback and replays the data at a certain ratio. Increase by 0.005 to form a new hybrid training set to enhance the playback strength of professional data.

[0033] The mental model fine-tuning system based on five-dimensional roles and closed-loop feedback of the present invention is implemented based on the mental model fine-tuning method and includes the following modules:

[0034] The character data generation module is used to construct five-dimensional character profiles and generate role-playing training data driven by prompt words;

[0035] The replay dataset construction module is used to randomly sample and synthesize a mixed training set according to the replay ratio;

[0036] The basic ability fine-tuning module is used to perform LoRA fine-tuning of the first stage of basic psychological abilities and generate the fine-tuned model. ;

[0037] The character style fusion module is used to optimize the model using a mixed training set. Perform the second stage of style fusion fine-tuning to generate the fine-tuned model. ;

[0038] The dual-track automated assessment module is used to call large-parameter, large-scale models to quantify the output results in terms of role and consultation dimensions.

[0039] The closed-loop feedback module is used to automatically execute capability enhancement strategies and perform iterative termination decisions based on the comparison results between the scoring results and the thresholds.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] First, addressing the issue of insufficient model approachability, this invention achieves a balance between professionalism and approachability through a data replay mechanism. In fine-tuning, this invention uses a hybrid training set of basic psychological data with dynamically proportioned data, enabling the model to master a more natural, conversational style (enhancing approachability) without losing its underlying psychological counseling logic. This effectively solves the problem of rigid dialogue caused by overly specialized data.

[0042] Secondly, addressing the issue of high costs in constructing high-quality role-playing data, this invention achieves low-cost data construction. This invention utilizes large-parameter, large-language models to replace human crowdsourcing in data generation, eliminating reliance on expensive manual coding, significantly reducing data construction costs, and enabling large-scale data production.

[0043] Third, addressing the lack of flexibility in existing role-playing methods, this invention constructs a role modeling method based on a five-dimensional structure. By defining "basic background, emotional characteristics, interaction patterns, relationship networks, and psychological support dimensions," this invention can quickly generate data suitable for different roles by simply adjusting the descriptive text, solving the problem that traditional methods struggle to quickly adapt to new scenarios.

[0044] Fourth, in response to the problem that existing technologies neglect practical, everyday scenarios, this invention uses a Cartesian product strategy to generate dialogue topics, ensuring that the model can handle real, relatable daily problems and solving the problem of technology being out of touch with actual user needs.

[0045] Fifth, to address the issue of insufficient objectivity in the evaluation scheme, this invention constructs a dual-track automated evaluation system and a closed-loop feedback mechanism. The dual-track automated evaluation eliminates the subjective bias of manual evaluation and automatically adjusts the sample size or data replay ratio based on whether the score reaches a preset threshold. The enhancement strategy ensures the determinism and efficiency of model optimization. Attached Figure Description

[0046] Figure 1 This is a flowchart of the present invention;

[0047] Figure 2 This is a schematic diagram illustrating the construction and data generation of the five-dimensional character profile in this invention;

[0048] Figure 3 This is a diagram of the dual-track automated evaluation system architecture in this invention;

[0049] Figure 4 This is a closed-loop feedback optimization decision logic diagram in this invention. Detailed Implementation

[0050] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0051] This invention provides a method for fine-tuning a mental model based on five-dimensional roles and closed-loop feedback, the complete process of which is as follows: Figure 1As shown, the system first generates initial training data by constructing a five-dimensional character profile. It then undergoes two stages: fine-tuning of basic abilities and fine-tuning of character style integration. A dual-track automated scoring and feedback mechanism is introduced, forming an automated closed loop from data generation to model optimization. This process ensures that the final psychological model achieves a balance between professionalism and approachability.

[0052] First, let's assume the base model to be fine-tuned is... .

[0053] Step 1: Generate role-playing training data based on the constructed five-dimensional character profile.

[0054] This step generates role-playing training data based on the constructed five-dimensional character profiles. The logic for generating the role-playing training data is detailed in the appendix. Figure 2 ,

[0055] The first step is to establish the five dimensions of the character, including basic background, emotional characteristics, interaction patterns, relationship network, and psychological support.

[0056] The five-dimensional character profile is defined as follows:

[0057] Basic background: Includes basic information such as age, gender, occupation, and educational background.

[0058] Emotional characteristics: Define the character's personality tone (such as optimistic, sensitive, serious) and common emotional response patterns.

[0059] Interactive Mode: Set the character's language style (e.g., colloquial, formal), speaking speed, and common catchphrases.

[0060] Relationship network: Define the strength of the relationship between roles and users (such as strangers, close friends, relatives) and the boundaries of interaction.

[0061] Psychological support dimension: Define the role's strategic preferences when providing psychological support (e.g., whether they prefer listening, giving advice, or emotional empathy).

[0062] The second step is to perform Cartesian product operations using the scene extension matrix module, such as... Figure 2 The dialogue seed module is shown in the image:

[0063]

[0064] In the formula Let m be a set of emotion lists. Given a set of n scene lists, combining them using a Cartesian product can produce a sequence of length n. Dialogue seed set .

[0065] The third step is to construct prompts that include a five-dimensional character profile, dialogue seeds, task instructions, and enforcement rules. The prompt template is as follows:

[0066] # Character Design (Persona)

[0067] You must strictly, completely, and immersively play the following role, and every word you utter must conform to the role's identity, tone, and knowledge background:

[0068] ---

[0069] {Please fill in the five-dimensional character profile mentioned above here}

[0070] ---

[0071] # Task Instruction

[0072] 1. Your core task is to engage in a natural, fluent, and in-depth multi-turn conversation with the user, in the role described above, on the topic of "{fill in a conversation seed mentioned above}".

[0073] 2. The entire dialogue must contain {num_turns} rounds of question and answer (1 round = 1 question + 1 answer).

[0074] 3. The conversation must be initiated by a user. In the generated JSON, the 'from' field of the first record must be 'human'.

[0075] 4. Follow-up questions must be closely related to the answers in the previous round, demonstrating the coherence of the dialogue.

[0076] 5. All your output must strictly adhere to the JSON format below and must not contain any additional explanations or text.

[0077] # Output Format

[0078] You must encapsulate the entire generated multi-turn dialogue in a JSON object. This JSON object should contain a key named "conversation_log" and a list whose value is a part of the dialogue, with each element having two keys: "from" and "value".

[0079] Example JSON structure:

[0080] {{

[0081] "conversation_log": [

[0082] {{ "from": "human", "value": ""}},

[0083] {{ "from": "gpt", "value": ""}},

[0084] {{ "from": "human", "value": ""}},

[0085] {{ "from": "gpt", "value": ""}} ]

[0087] }}

[0088] Using the above prompts, role-playing training data is generated, resulting in a role-playing training dataset. This process significantly improves the efficiency of data construction.

[0089] Step 2: Construct a hybrid training set

[0090] This step involves randomly selecting a portion of the basic psychological counseling dataset at a specific ratio, replaying the role-playing training data, and obtaining a mixed training set.

[0091] The first step is to calculate the amount of replay data. : ,in This represents the initial data replay ratio, with a value of 0.01. This represents the total amount of data in the basic psychological counseling dataset;

[0092] The second step involves randomly selecting samples from the basic psychological counseling dataset. The data points are shuffled and mixed with the role-playing training data generated in step 1 to obtain a mixed training set.

[0093] Step 3: Fine-tuning basic skills

[0094] Using a basic psychological counseling dataset to model the base model LoRA fine-tuning was performed to obtain the model after the first fine-tuning. The basic psychological counseling dataset can be a publicly available mental health instruction fine-tuning dataset, such as, but not limited to, one or more datasets like Smile, PsyQA, or CPD. An open-source large language model is used as the model to be fine-tuned, LoRA adapter parameters are configured, and training is performed using the publicly available mental health instruction fine-tuning dataset at a preset learning rate to obtain a first-stage fine-tuned model with a professional psychological kernel. .

[0095] Step 4: Fine-tuning the character style

[0096] The fine-tuned model was trained using a mixed training set. LoRA was used to fine-tune the character's style, resulting in the adjusted model. The fine-tuned model As the model to be fine-tuned, the model used in step three is adopted. Using the same learning rate, while preserving the model's psychological expertise, the model was given a specific character expression style, resulting in a second fine-tuned model. .

[0097] Step 5: Dual-track automated scoring

[0098] Using a large model with many parameters to fine-tune the model after the second round of adjustments The batch output results are scored using a dual-track system, as shown in Figure 3:

[0099] The first step involves using a large model with a large number of parameters to score the role-playing dimension from three aspects: consistency, anthropomorphism, and attractiveness. ;

[0100] The second step involves using a large-parameter, high-model approach to score the dimensions of psychological counseling from three aspects: emotional empathy, cognitive empathy, and guidance ability. ;

[0101] The third step is to adjust the weighting coefficients. and Calculate the total score ,in .

[0102] Step 6: Execution Feedback

[0103] The scoring results trigger a closed-loop feedback mechanism. As shown in Figure 4, a preset threshold is set to 90% of the model's maximum scoring value.

[0104] like If the model falls below a preset threshold, indicating a lack of character consistency, anthropomorphism, or appeal, the system will automatically trigger feedback and execute the formula. After increasing the sample size of the role-playing training data, a new mixed training set is generated, and step 4 is executed again.

[0105] like If the level falls below a preset threshold, indicating catastrophic amnesia leading to reduced emotional empathy, cognitive empathy, and guidance abilities, the system automatically triggers feedback and executes the formula. Increase the proportion of data replay in the mixed training set. Repeat step 4.

[0106] Step 7: Determine if the iteration terminates

[0107] Determine whether both the individual dimension score and the total score have reached the corresponding threshold or the preset maximum number of feedback rounds. If so, the trained mental model is obtained; otherwise, continue to step 5. It should be noted that this is a re-scoring of the batch-generated results of the model fine-tuned in the second stage, but the scoring process is consistent with the dual-track automated scoring described above.

[0108] Example

[0109] This embodiment takes the construction of a large-scale psychological health model with the style of a "college student roommate" as an example to provide a detailed explanation of the psychological model fine-tuning method based on five-dimensional roles and closed-loop feedback provided by the present invention.

[0110] Base model selection: The Qwen-7B-Instruct model was selected as the base model to be fine-tuned. .

[0111] Step 1: Generate role-playing training data

[0112] The first step is to establish the five dimensions of the role: defining the five-dimensional role profile of a "college student roommate":

[0113] Basic background: College students aged 20-22 are facing pressure from postgraduate entrance examination and employment.

[0114] Emotional characteristics: The overall tone is cheerful but occasionally anxious, which can resonate with peers.

[0115] Interactive mode: Frequently uses campus vocabulary such as "8 a.m." and "failing a course", speaks quickly, and avoids lecturing.

[0116] Relationship Network: Set up as the user's close roommates, supporting late-night chats.

[0117] Psychological support dimension: a preference for adopting a strategy of "complaining and empathizing first, then offering advice".

[0118] The second step is scene expansion, setting the quantity to [number]. The list of emotions (such as anxiety, anger, etc.) and the number of them are The list of scenarios (such as heartbreak, failing a course warning, etc.) is used to perform Cartesian product operations. This generates a set of dialogue seeds of length 500.

[0119] The third step is to concatenate the prompts to generate role-playing data: construct prompt words containing the aforementioned five-dimensional profile, dialogue seed, task instructions, and JSON format requirements, and drive the large language model. In this embodiment, the Deepseek-v3.2 model is used for batch generation. The role-playing training data forms a dataset. .

[0120] Step 2: Construct a hybrid training set

[0121] The first step is to calculate the amount of replay data: In this embodiment, the SoulChat dataset is used, and the initial data replay ratio is set. Basic Psychological Counseling Dataset The total amount According to the formula Calculate: Le .

[0122] The second step is to synthesize a mixed training set: randomly select 300 data points from the SoulChat dataset and combine them with the 2000 role-playing training data points generated in step 1. Shuffle and mix to obtain a mixed training set.

[0123] Step 3: Fine-tuning basic skills

[0124] Configure the LoRA adapter parameters and train the model using 20,000 SoulChat data points at a learning rate of 2e-5. Fine-tuning was performed to give the model basic professional psychological counseling capabilities, resulting in the model after the first fine-tuning. .

[0125] Step 4: Fine-tuning the character style

[0126] The finely tuned model As the model to be fine-tuned, the same learning rate as in step 3 was used, and LoRA fine-tuning of character style fusion was performed using the mixed training set obtained in step 2. While preserving the model's psychological expertise, the model was made to acquire the expressive style of a "college student roommate," resulting in the model after the second fine-tuning. .

[0127] Step 5: Dual-track automated scoring

[0128] Using GPT-4 as the referee for a large model with many parameters, weight coefficients were set. and ,right Scoring is performed on the batch output results:

[0129] Step 1: Assess consistency, anthropomorphism, and appeal to obtain a role-playing dimension score. .

[0130] Step 2: Assess emotional empathy, cognitive empathy, and guidance abilities to obtain scores across the psychological counseling dimensions. .

[0131] Step 3: Calculate the total score .

[0132] Step 6: Execution Feedback

[0133] Set the preset threshold to 90% of the upper limit (i.e., 4.5 points):

[0134] because If the value is below the preset threshold of 4.5, the model is deemed to lack sufficient anthropomorphism. The system automatically triggers feedback and executes the formula. ,Right now At this point, an additional 300 role-playing training data points need to be generated using the Deepseek-v3.2 model, and a new mixed training set needs to be generated according to step 2. Then, the style fusion fine-tuning in step 4 is performed again.

[0135] Step 7: Determine if the iteration terminates

[0136] Second round of evaluation: Execute step 5 again to obtain... , Calculate the total score Both the individual dimension scores and the total score reached the preset threshold of 4.5. The system terminated the feedback and finally output a well-trained mental model that simultaneously possesses basic mental health capabilities and the specific expressive style of a "college student roommate".

Claims

1. A method for fine-tuning a mental model based on five-dimensional roles and closed-loop feedback, characterized in that, The steps of this fine-tuning method include the following: Step 1: Generate role-playing training data based on the constructed five-dimensional structured character profile system; Step 2: Randomly select a portion of the basic psychological counseling dataset according to a specific ratio, and replay the role-playing training data to obtain a mixed training set; Step 3: Use the basic psychological counseling dataset to fine-tune the base model using LoRA to obtain the fine-tuned model. ; Step 4: Use a mixed training set to fine-tune the model. LoRA was fine-tuned by blending character styles to obtain the adjusted model. ; Step 5: Use a large model with many parameters to fine-tune the model. The batch output results are scored using a dual-track system, based on both the "role-playing dimension" and the "psychological counseling dimension". Step 6: If the score for the role-playing dimension is lower than its preset threshold, increase the sample size of the role-playing training data in the mixed training set to generate a new mixed training set, and then repeat Step 4; if the score for any item in the psychological counseling dimension is lower than its preset threshold, increase the data replay ratio in the mixed training set. After generating a new mixed training set, repeat step 4; Step 7: Determine whether the individual dimension score and the total score have both reached the corresponding threshold or the preset maximum number of feedback rounds. If so, the trained mental model is obtained; otherwise, proceed to step 5.

2. The method for fine-tuning the mental model according to claim 1, characterized in that, The structured character profile system built on five dimensions, as described in Step 1, includes basic background, emotional characteristics, interaction patterns, relationship network, and psychological support. Basic background includes the age, gender, occupation, and educational background of the virtual character to be portrayed. Emotional characteristics refer to the personality tone and common emotional response patterns of the virtual character. Interaction patterns refer to the language style, speaking speed, and common catchphrases of the virtual character. Relationship network refers to the strength of the relationship and interaction boundaries between the virtual character and the user. Psychological support refers to the strategic preferences of the virtual character when providing psychological support. The virtual character to be portrayed is a virtual personality with specific social identity, psychological traits, and language habits, concretely simulated according to the needs of the consultation scenario through the parameterized definition of the above five dimensions; it is not a specific natural person.

3. The method for fine-tuning the mental model according to claim 1, characterized in that, The role-playing training data mentioned in step 1 refers to a JSON-formatted dataset that includes five-dimensional character profiles, dialogue topics, task instructions, and prompts for mandatory rules, which drives the large language model to generate the dataset.

4. The method for fine-tuning the mental model according to claim 1, characterized in that, The steps described in step 2, which involve randomly selecting a portion of the basic psychological counseling dataset at a specific ratio and replaying the role-playing training data, are as follows: The first step is to calculate the amount of replay data. : ,in, The initial data replay ratio is 0.

01. This represents the total amount of data in the basic psychological counseling dataset; The second step involves randomly selecting samples from the basic psychological counseling dataset. The data is shuffled and mixed with the role-playing training data generated in step 1 to obtain a mixed training set for replaying the role-playing training data.

5. The method for fine-tuning the mental model according to claim 1, characterized in that, The steps for fine-tuning the base model using the basic psychological counseling dataset in step 3 are as follows: The first step is to use the open-source large language model as the model to be fine-tuned in this step; The second step involves configuring the LoRA adapter parameters and training the model using a publicly available mental health instruction fine-tuning dataset at a preset learning rate to obtain a fine-tuned model with a professional psychological core. .

6. The method for fine-tuning the mental model according to claim 5, characterized in that, Step 4 describes using a mixed training set to fine-tune the model. The steps for fine-tuning LoRA's character style are as follows: The first step is to fine-tune the model. This serves as the model to be fine-tuned in this step; The second step involves using a fine-tuning model. At the same learning rate, while preserving the model's psychological expertise, LoRA fine-tunes the model to obtain a specific character expression style, resulting in a fine-tuned model. .

7. The method for fine-tuning the mental model according to claim 1, characterized in that, The steps in step 5, which involve using a large model with many parameters to score the "role-playing dimension" and the "psychological counseling dimension" on a dual-track basis, are as follows: The first step is to fine-tune the model for the current round using a large number of parameters and a large model. The batch output results were scored based on three aspects: consistency, anthropomorphism, and attractiveness, to determine the "role-playing dimension." , The range of values ​​is ; The second step is to fine-tune the model for the current round using a large model with many parameters. The batch output results are scored from three aspects: emotional empathy, cognitive empathy, and guidance ability, to obtain the "psychological counseling dimension". , The range of values ​​is ; The third step is to apply the preset weighting coefficients. and Calculate the total score ,in, and The range of values ​​is .

8. The method for fine-tuning the mental model according to claim 7, characterized in that, The "role-playing dimension score below its preset threshold" mentioned in step 6 means that the preset threshold is 90% of the model's maximum score. When the values ​​fall below a threshold, the model is deemed to have insufficient role consistency, anthropomorphism, or attractiveness. In this case, the system automatically triggers feedback, adding 300 newly generated role-playing training data points to the mixed training set to form a new mixed training set, thereby enhancing the role-playing ability.

9. The method for fine-tuning the mental model according to claim 7, characterized in that, In step 6, the score of a single item in the psychological counseling dimension being lower than its preset threshold means that the preset threshold is 90% of the upper limit of the model score. When the data falls below a certain threshold, the system determines that the model has experienced catastrophic amnesia, leading to a decline in emotional empathy, cognitive empathy, and guidance capabilities. At this point, the system automatically triggers feedback and replays the data at a certain ratio. Increase by 0.005 to form a new hybrid training set to enhance the playback strength of professional data.

10. A mental model fine-tuning system based on five-dimensional roles and closed-loop feedback, characterized in that, This method, based on any one of the mental model fine-tuning methods described in claims 1-9, includes the following modules: The character data generation module is used to construct five-dimensional character profiles and generate role-playing training data driven by prompt words; The replay dataset construction module is used to randomly sample and synthesize a mixed training set according to the replay ratio; The basic ability fine-tuning module is used to perform LoRA fine-tuning of the first stage of basic psychological abilities and generate the fine-tuned model. ; The character style fusion module is used to optimize the model using a mixed training set. Perform the second stage of style fusion fine-tuning to generate the fine-tuned model. ; The dual-track automated assessment module is used to call large-parameter, large-scale models to quantify the output results in terms of role and consultation dimensions. The closed-loop feedback module is used to automatically execute capability enhancement strategies and perform iterative termination decisions based on the comparison results between the scoring results and the thresholds.

Images