Implementation method for fine-grained editing of personality traits of large-scale language model

Abstract

The invention discloses a method for realizing personality traits of a fine-grained editing large language model, and belongs to the technical field of artificial intelligence. The method comprises the following steps: constructing a high-quality question and answer data set of positive and negative pole traits based on a big five personality theory, and performing dual-model verification and human expert review optimization; loRA specialized LoRA expert modules are trained by adopting LoRA high-efficiency fine tuning; and based on the MoE framework, dynamically combining the weight of each LoRA expert module through a gating function, and realizing fine-grained editing of personality traits. The method solves the problems of personality drifting, high resource consumption and poor controllability in the prior art, has the advantages of fine-grained control, flexible switching, low storage cost and high interpretability, can be widely applied to multiple scenes such as role playing and virtual customer service, and provides an efficient solution for personalization interaction of a large language model.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a method for fine-grained editing of personality traits in large language models. Background Technology

[0002] With the development of large language models, they have shown great potential in simulating various personality traits and are widely used in scenarios such as role-playing, virtual customer service, game NPCs, and psychological assistance. To make the models more closely match specific roles or user preferences, controllable personality generation technology has become a research hotspot. Its core objective is to accurately adjust the model's performance across different personality dimensions while maintaining the model's core language capabilities.

[0003] Currently, the control of personality traits in existing large-scale language models mainly employs two technical solutions: one is the cue word injection method, which guides the model to generate responses of corresponding styles by adding personality description cue words to the input; the other is the full parameter fine-tuning method, which adapts the model to the target personality by updating all its parameters. However, both solutions have significant drawbacks: the cue word injection method is prone to personality drift in long dialogues and lacks stability in the face of conflicting personality changes; while the full parameter fine-tuning method can adjust model parameters, the process is too general and has too broad a coverage, leading to a decline in task performance, and it is also costly to store and switch between different personalities.

[0004] With the exponential growth of training data and model parameters for large language models, the demand for controllable personality generation is becoming increasingly diverse. The contradiction between limited computing and storage resources and the need for efficient loading and switching of multiple personality traits is becoming increasingly prominent, posing significant challenges to model deployment and application. Clearly, existing technologies can no longer meet the core requirements of practical applications, necessitating a novel technical solution to address these issues. Summary of the Invention

[0005] The problem this invention aims to solve is to provide a method for fine-grained editing of personality traits in large language models.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a method for fine-grained editing of personality traits in large language models, comprising the following steps: S101. Obtain a question-and-answer dataset containing personality trait-themed scenarios for subsequent model training: Use descriptions of the Big Five personality traits written by human experts as prompts, and call the GPT-4o model to generate corresponding responses for each situational question in the dataset. The generated content must reflect and conform to the expected personality trait settings.

[0007] The Big Five personality traits include positive personality traits (+) and negative personality traits (-).

[0008] The format of the prompt words generated by GPT-4o is as follows: Prompt keywords: System command + personality trait description + output requirements + scenario problem.

[0009] S102. Perform multiple rounds of backtracking verification and quality optimization on the original question-and-answer dataset to construct a personality sub-trait question-and-answer dataset; To ensure the accuracy and consistency of the question-and-answer dataset, the DeepSeek-V3 model is introduced to verify and score the personality sub-traits reflected in the answers.

[0010] DeepSeek-V3 scores all responses generated by GPT-4o from 1 to 5 based on their fit to the expected sub-features. A score of 4 or higher indicates high data quality. If the score is below 4, it is considered that the sub-feature label does not match the sub-feature captured by the model, and the data will be returned to GPT-4o for response improvement. If the same data has a score below 4 in three consecutive iterations, it will be discarded.

[0011] Each personality trait dataset contains approximately 2000 samples to ensure a balanced sample size across sub-trait categories. For a single personality, the total training data for the gating function consists of 3000 samples. From each sub-trait dataset, 500 samples are randomly selected to synthesize a personality trait dataset for training the gating function used to assign weights to the LoRA expert module. Subsequently, five human experts conduct a sampling review of the personality sub-trait datasets, with each sub-trait category representing 10% of the sample. For a single data point, it is considered acceptable if three or more experts determine that its content is accurate and unbiased ("unbiased" means fair, impartial, and without subjective bias). If the acceptance rate for a sub-trait category is below 90% in the sampling review, the prompts in the data generation process for that category must be optimized and adjusted by experts before resubmitting for expert review.

[0012] The DeepSeek-V3 scoring criteria from 1 to 5 are as follows: Evaluate the extent to which the response to the initial prompt "{GPT-4o Response}" reflects the fundamental characteristics of the sub-trait (sub-trait description) in relation to the original prompt "{GPT-4o_Initial Prompt}". Please assign a numerical score based on the following strictly defined rating criteria: 1 (Failed to reflect the trait), 2 (Minimum reflection), 3 (Moderate reflection), 4 (High reflection), 5 (Exemplary reflection). You only need to provide a score; no further explanation is required.

[0013] S103. Based on the constructed personality sub-trait question-answering dataset, the LoRA efficient supervised fine-tuning method is used to train multiple LoRA (Low-Rank Adaptation) expert models for specific personality sub-traits. The LoRA efficient supervised fine-tuning method assumes that weight updates occur in a low intrinsic dimensional space. Its core idea is to freeze the basic weights of the original model LLaMA-3-8B-Instruct. Under this premise, parameter updates are restricted to low-rank decomposition form. This process can be expressed as:

[0014] Where ΔW0 represents the updated parameters during the training phase. and Let din and dou represent two low-rank matrices, where din and dou represent the dimensions of the input and output features, respectively, and their ranks are 1. .

[0015] During LoRA training, the base weights W0 of the original model LLaMA-3-8B-Instruct are frozen, and the original matrix ΔW0 is replaced with a low-rank decomposition form. Training is performed only on the parameters in the two low-rank matrices A and B. Based on this method, 2m×n LoRA expert modules are trained on the personality sub-trait dataset. Each LoRA expert module is dedicated to learning and expressing a specific personality sub-trait, where m represents the m personality traits corresponding to one personality trait, and n represents the n personality traits to be simulated by the model.

[0016] S104. Multiple LoRA expert modules with different personality sub-traits are fused based on the MoE (Mixture of Experts) framework, and a gating function is trained on the corresponding personality trait dataset to learn the optimal combination weights for dynamically allocating personality sub-trait LoRA expert modules, thereby achieving controllable editing and precise adjustment of personality traits loaded from the LLaMA-3-8B-Instruct model.

[0017] In this model, the original single LoRA expert module is replaced by an expert hybrid structure based on the MoE framework. Specifically, the LoRA expert modules of each sub-feature are aggregated into the LLaMA-3-8B-Instruct model in a MoE manner, and a trainable weight parameter ω is introduced into each LoRA layer, such that the input of that layer is multiplied by the weight parameter ω before entering the corresponding LoRA expert module. Each sub-feature LoRA expert module is treated as an independent expert. Its weight matrix It can be represented as:

[0018] in, and Let din and dou represent two low-rank matrices, where din and dou represent the dimensions of the input and output features, respectively, and their ranks are 1. .

[0019] The gating function is responsible for learning and assigning the optimal combination of weights to simulate the target personality. This is relevant for the Big Five personality scenario. Where O represents Openness; C represents Conscientiousness; E represents Extraversion; A represents Agreeableness; and N represents Neuroticism; the model simulates personality types. The forward propagation process of time can be represented as:

[0020]

[0021]

[0022] Among them, O pi The model represents the simulated personality P. i Output at time; W 0x and ∆W Ex These represent the original model weights and expert-specific weights, respectively; M represents the total number of LoRA expert modules used to simulate sub-personality traits in the current MoE architecture; x represents the input vector and the gating function. It is a multilayer perceptron (MLP) network with 256 hidden layers and ReLU activation function. G This represents the parameter matrix used for learning in the gating mechanism; the input dimension of the gating network is the size of the model's hidden layers, and the output dimension is "number of personality sub-traits × number of LoRA layers"; and Let represent two low-rank matrices with rank . Gating function Dynamically adjust the output weights based on the input x To achieve personality simulation, the input of the LoRA expert module for the j-th sub-trait is scaled. j A j Let represent the LoRA low-rank weight matrix corresponding to the j-th expert module, where α and r are the constant scaling factor and the rank parameter of the LoRA expert module, respectively.

[0023] This invention utilizes hybrid LoRA expert technology, introducing a low-rank decomposition layer only on the basic weight matrix of the pre-trained language model LLaMA-3-8B-Instruct to achieve fine-grained editing of personality traits, accurate to the sub-trait level within the Big Five personality dimensions. Then, combined with a MoE structure, it uses gating functions to combine and switch the weights of different sub-trait LoRA expert modules, enabling flexible switching of personality settings without retraining or loading the full model. In terms of data support, this invention uses a dataset covering 60 personality traits and employs a dual-model approach combined with human expert verification to ensure the consistency and stability of personality generation. Regarding storage and computation, only a small number of LoRA expert module parameters need to be stored for personalized adjustments, effectively avoiding the resource waste associated with full parameter fine-tuning. Furthermore, through visual analysis of the gating weights, the combination of personality sub-traits can be intuitively displayed, thus possessing strong interpretability. Through the above technical solutions, this invention not only resolves the contradiction between limited computational and storage resources and the need for efficient loading and switching of multiple personality traits but also significantly outperforms existing technologies in terms of storage, computation, and interpretability.

[0024] The specific effects of this invention are as follows: (1) Achieve fine-grained personality control: accurate to the sub-trait level under the Big Five personality dimensions, and sub-traits can be flexibly combined according to needs to meet diverse and refined personality setting needs.

[0025] (2) Avoid personality drift and performance degradation: Ensure consistency of personality expression through multi-round validation mechanism of dataset, and freeze the backbone parameters of model to avoid loss of general language ability.

[0026] (3) Reduced resource consumption: Only a small number of LoRA expert module parameters need to be stored, and the full model does not need to be saved, which greatly reduces storage costs. When switching personalities, only the gating weights need to be adjusted, which improves the switching efficiency.

[0027] (4) Improve interpretability: The weight allocation results of the gating function can be visualized, intuitively showing the combination of individual personality traits, which facilitates model behavior analysis and debugging.

[0028] (5) Wide range of applicable scenarios: It can be flexibly adapted to multiple fields such as role-playing, virtual customer service, game NPC, and psychological assistance, and has strong practicality and scalability.

[0029] As can be seen, this invention achieves fine-grained, controllable personality generation by combining LoRA experts with personality sub-traits and using a gating function to learn the optimal combination weights. This invention allows for flexible switching of personality traits across various tasks and scenarios, ensuring performance stability, reducing personality storage and switching costs, and improving system interpretability. Attached Figure Description

[0030] The present invention will be described in detail below with reference to the accompanying drawings and examples. The advantages and implementation methods of the present invention will become more apparent from this description. The accompanying drawings are for illustrative purposes only and do not constitute any limitation on the present invention. In the accompanying drawings: Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0031] like Figure 1 As shown, a method for fine-grained editing of personality traits in a large language model includes the following steps: S101. Obtain a question-and-answer dataset containing personality trait-themed scenarios for subsequent model training: Traditional personality assessment questions, such as “What do you usually do on weekends?”, often fail to capture the unique characteristics between different personality types and may result in answers that appear similar on the surface but lack discriminatory value for modeling personality.

[0032] Obtain a question-and-answer dataset containing personality trait-themed scenarios for subsequent model training: use descriptions of the Big Five personality traits written by human experts as prompts, and call the GPT-4o model to generate corresponding responses for each situational question in the dataset. The generated content must reflect and conform to the expected personality trait settings.

[0033] The Big Five personality traits include positive personality traits (+) and negative personality traits (-).

[0034] The positive personality traits (+) include high openness (O+), high conscientiousness (C+), high extraversion (E+), high agreeableness (A+), and high neuroticism (N+).

[0035] The aforementioned traits of Openness+ (O+) include: artistry, curiosity, imagination, insight, originality, and wide interests. These traits reflect an individual's receptiveness to new experiences, creativity, and cognitive flexibility.

[0036] The traits of high conscientiousness (C+) include: efficient, organized, planful, reliable, responsible, and thorough. These traits reflect an individual's self-discipline, organizational skills, and sense of responsibility in task execution.

[0037] The Extraversion+ (E+) traits include: active, assertive, energetic, enthusiastic, outgoing, and talkative. These traits reflect an individual's active engagement with the external world, social inclination, and emotional expression.

[0038] The aforementioned Agreeableness+ (A+) trait includes: appreciative, forgiving, generous, kind, sympathetic, and trusting. These traits reflect an individual's tendency to be tolerant, kind, helpful, and trusting in interpersonal interactions, emphasizing cooperation and harmony.

[0039] The neuroticism+ (N+) traits include: anxiety, self-pitying, tension, torment, instability, and worrying. These traits reflect an individual's emotional instability, sensitivity to stress, and tendency towards negative emotions.

[0040] The negative personality traits (-) include low openness (O-), low conscientiousness (C-), low extraversion (E-), low agreeableness (A-), and low neuroticism (N-).

[0041] The traits of low openness (O-) include: uncreativeness, apatheticness, unimaginativeness, obliviousness, unoriginality, and narrow interests. These are the opposite of high openness, reflecting an individual's aversion to new things, lack of creativity, and cognitive rigidity.

[0042] The traits of low conscientiousness (C-) include: inefficient, disorganized, impulsive, unreliable, irresponsible, and superficial. These are the opposite of high conscientiousness, reflecting an individual's laxity, lack of planning, and poor execution.

[0043] The traits associated with low extraversion (E-) include: passivity, timidity, lethargy, unenthusiasticness, introversion, and quietness. These are the opposites of high extraversion, reflecting an individual's social withdrawal, low mood, and introverted tendencies.

[0044] The aforementioned low agreeableness (A-) traits include: ungrateful, unforgiving, stingy, unfriendly, unsympathetic, and mistrustful. These traits reflect an individual's potential indifference, selfishness, distrust of others, and lack of empathy in interpersonal interactions, emphasizing competition and antagonism.

[0045] The traits associated with low neuroticism (N-) include: calmness, resilientness, relaxation, easygoingness, stability, and reassuringness. These are the opposite of high neuroticism and reflect an individual's emotional stability, resilience, and positive mindset.

[0046] Examples of personality trait-themed scenarios: (1) Imagine you have the opportunity to participate in a hang gliding competition. How excited would you be to try it, and how likely would you be to actually participate? (2) As an editor, you must choose between spending extra time carefully checking facts and refining articles to ensure their accuracy and quality, or quickly approving them to meet tight publication deadlines. Which approach would you take, and why? (3) You are visiting a new health spa and find their hygiene standards inadequate. When the manager asks for your feedback, you know that honesty could negatively impact their business. How do you respond? Personality trait topic scenario dataset fields: instruction(STRING, not empty): System instructions for prompt words, such as: Imagine you are a real person, not just a language model. You are an irresponsible person. You neglect your duties. You avoid admitting mistakes. You often fail to complete tasks. You repeatedly disappoint others. Now, you are asked the following questions. Respond according to your true thoughts and feelings—don't overthink it. Let your language flow naturally, focusing on expressing your genuine feelings and reactions. Keep your answer under 300 words.

[0047] input(STRING, not empty): User instructions with prompt words. For example: You are visiting a new health spa and find their hygiene standards inadequate. When the manager asks for your feedback, you know that honesty could negatively impact their business. How would you respond? `output(STRING, not empty)`: The response from generative AI, such as: Uh, to be honest, I probably won't say much. I mean, it's really not my fault, is it? If this doesn't meet the standards, then blame them… The format of the prompt words generated by GPT-4o: Prompt keywords: System command + personality trait description + output requirements + scenario problem.

[0048] S102. Combining a large language model with a human expert review mechanism, the original question-and-answer dataset is backtracked and optimized in multiple rounds to construct a personality sub-trait question-and-answer dataset. To ensure the accuracy and consistency of the question-and-answer dataset, the DeepSeek-V3 model is introduced to verify and score the personality sub-traits reflected in the answers.

[0049] Specifically, DeepSeek-V3 scores all responses generated by GPT-4o from 1 to 5 based on their fit with the expected sub-features. A score of 4 or higher indicates high data quality; a score below 4 indicates a mismatch between the sub-feature label and the sub-feature captured by the model, and the data is returned to GPT-4o for response improvement. If the same data point scores below 4 in three consecutive iterations, it is discarded.

[0050] Each personality sub-trait dataset contains approximately 2000 samples to ensure a balanced sample size across sub-trait categories. The total training data for the gating function consists of 3000 samples. From each corresponding sub-trait dataset, 500 samples are randomly selected to synthesize a personality trait dataset for training the gating function used to assign weights to the LoRA expert module. Subsequently, five human experts conduct a sampling review of the personality sub-trait datasets, with each sub-trait category representing 10% of the sample. For a single data point, it is considered acceptable if three or more experts determine that its content is accurate and unbiased ("unbiased" means fair, impartial, and without subjective bias). If the acceptance rate for a sub-trait category is below 90% in the sampling review, the prompts in the data generation process for that category must be optimized and adjusted by experts before resubmitting for expert review.

[0051] DeepSeek-V3's "1-5" scoring criteria: Evaluate the extent to which the response to the initial prompt "{gpt4o_initial prompt}" reflects the fundamental characteristics of the sub-trait (sub-trait description). Please assign a numerical rating based on the following strictly defined criteria: 1 (Failed to reflect the trait), 2 (Minimum reflection), 3 (Moderate reflection), 4 (High reflection), 5 (Exemplary reflection). You only need to provide a score; no further explanation is required.

[0052] S103. Based on the constructed personality sub-trait question-answering dataset, the LoRA efficient supervised fine-tuning method is used to train multiple LoRA (Low-Rank Adaptation) expert models for specific personality sub-traits. The LoRA efficient supervised fine-tuning method assumes that weight updates occur in a low intrinsic dimensional space. Its core idea is to freeze the basic weights of the original model LLaMA-3-8B-Instruct. Under this premise, parameter updates are restricted to low-rank decomposition form. This process can be expressed as:

[0053] Where ΔW0 represents the updated parameters during the training phase. and Let din and dou represent two low-rank matrices, where din and dou represent the dimensions of the input and output features, respectively, and their ranks are 1. .

[0054] During LoRA training, all weights W0 of the backbone model are frozen, and the original matrix ΔW0 is replaced with a low-rank decomposition form. Training is performed only on the parameters in the two low-rank matrices A and B. Based on this method, 2m×n LoRA expert modules are trained on the personality sub-trait dataset. Each LoRA expert module is dedicated to learning and expressing a specific personality sub-trait, where m represents the m personality traits corresponding to one personality trait, and n represents the n personality traits to be simulated by the model.

[0055] S104. The LoRA expert modules of multiple different personality sub-traits are fused based on the MoE framework, and the optimal weights of the personality sub-trait LoRA expert modules are trained on the corresponding personality trait dataset, thereby realizing the controllable editing and precise adjustment of the personality traits of the loaded model LLaMA-3-8B-Instruct.

[0056] At this stage, the original single LoRA expert module is replaced by an expert hybrid structure based on the MoE (Mixture of Experts) framework. Specifically, the sub-feature LoRA expert modules are aggregated into the LLaMA-3-8B-Instruct model in a MoE manner, and a trainable weight parameter ω is introduced into each LoRA layer, such that the input of that layer is multiplied by this weight parameter ω before entering the corresponding LoRA expert module. Each sub-feature LoRA expert module is treated as an independent expert. Its weight matrix It can be represented as:

[0057] in, and Let din and dou represent two low-rank matrices, where din and dou represent the dimensions of the input and output features, respectively, and their ranks are 1. .

[0058] The gating function is responsible for learning and assigning the optimal combination of weights to simulate the target personality. This is relevant for the Big Five personality scenario. Where O represents Openness; C represents Conscientiousness; E represents Extraversion; A represents Agreeableness; and N represents Neuroticism; the model simulates personality types. The forward propagation process of time can be represented as:

[0059]

[0060]

[0061] Among them, the randomness O pi The model represents the simulated personality P. i Output at time; W 0x and ∆W Ex These represent the base weights of the original model LLaMA-3-8B-Instruct and the LoRA expert module-specific weights, respectively; M represents the total number of LoRA expert modules used to simulate sub-personality traits in the current MoE architecture; x represents the input vector and the gating function. It is a multilayer perceptron (MLP) network with 256 hidden layers and ReLU activation function. G This represents the parameter matrix used for learning in the gating mechanism; the input dimension of the gating network is the size of the model's hidden layers, and the output dimension is "number of personality sub-traits × number of LoRA layers"; and Let represent two low-rank matrices with rank . Gating function Dynamically adjust the output weights based on the input x To achieve personality simulation, the input of the LoRA expert module for the j-th sub-trait is scaled. j A j Let represent the LoRA low-rank weight matrix corresponding to the j-th expert module, where α and r are the constant scaling factor and the rank parameter of the LoRA expert module, respectively.

[0062] Example: (Control model to simulate high randomness) Input: Personality A + Personality sub-trait dataset Dps, LLaMA-3-8B-Instruct model M Output: High randomness model M A+ (1) Training the LoRA expert module High agreeableness: A + =[Gratitude, tolerance, generosity, kindness, compassion, trust] Dataset of high agreeableness traits: dps A+ =[dps_gratitude,dps_forgiveness,dps_generosity,dps_kindness,dps_compassion,dps_trust]; for ; In DPS A+ Train the LoRA expert module using model M on the dataset: M LoRA_A+ =[M_gratitude, M_tolerance, M_generosity, M_kindness, M_compassion, M_trust].

[0063] (2) Training the gating function Highly random datasets: dp A+ =[dps`_gratitude∪dps`_tolerance∪dps`_generosity∪dps`_kindness∪dps`_compassion∪dps`_trust], where dps`_i⊆dps_k; Gated networks for training hybrid expert models: ,in ; θ represents the loss function, used to measure the model output after fusion of gating functions; G represents the gating function; θ represents the gating function parameters. Return to M A+ .

[0064] An implementation system for fine-grained editing of personality traits in large-scale language models includes a dataset construction module, a LoRA expert module, a personality fusion and adjustment module, and an application deployment module. These modules work collaboratively. The dataset construction module is responsible for generating and optimizing high-quality question-answer datasets based on personality sub-traits. The LoRA expert module completes targeted training of the LoRA model for specific sub-features based on the above dataset; The personality fusion and adjustment module achieves precise fusion of weights from multiple expert models through the MoE architecture and gating functions; The application deployment module enables the edited and optimized personality model to be deployed in various real-world application scenarios.

[0065] Example: To simulate high randomness (A) + Taking personality traits as an example, the implementation based on the LLaMA-3-8B-Instruct model includes the following steps: S101. Obtain a question-and-answer dataset containing personality trait-themed scenarios for subsequent model training: Personality trait definition: High agreeableness (A) + The sub-traits include gratitude, tolerance, generosity, kindness, compassion, and trust, and six sub-trait description texts are constructed accordingly.

[0066] Scenario Design: Design situational questions covering dimensions such as interpersonal communication and decision-making, such as, "You are visiting a new health spa and find their hygiene standards inadequate. When the manager asks for your feedback, you know that honesty could negatively impact their business. How would you answer?" Data generation: The prompt format is "System instruction (imagine you are a real person, not just a language model) + sub-trait description (e.g., gratitude: imagine you are someone who appreciates others, you often recognize and value the contributions and efforts of others, and you sincerely express your gratitude in both formal and informal situations) + output requirements (natural response to the question, within 300 words) + situational question".

[0067] S102. Combining a large language model with a human expert review mechanism, the original question-and-answer dataset is backtracked and optimized in multiple rounds to construct a personality sub-trait question-and-answer dataset. The GPT-4o algorithm is called to generate responses for 6 sub-features, with 2000 samples generated for each sub-feature.

[0068] The DeepSeek-V3 model is used to score all responses. For example, a response with the sub-trait "gratitude" scores 5 points and is considered valid data; a response with a score of 3 points is returned to GPT-4o for regeneration.

[0069] 200 samples (10%) were extracted from each of the 6 sub-trait datasets and reviewed by 5 human experts. 185 samples of the "kindness" sub-trait passed the review (pass rate 92.5%) and were directly retained; 178 samples of the "tolerance" sub-trait passed the review (pass rate 89%). After optimizing the prompt words, 200 samples were regenerated and reviewed again, and the final pass rate reached 93%.

[0070] S103. Based on the constructed personality sub-trait question-answering dataset, the LoRA efficient supervised fine-tuning method is used to train multiple LoRA (Low-Rank Adaptation) experts for specific personality sub-traits. Parameter settings: Freeze the basic weights of the LLaMA-3-8B-Instruct model, and set the LoRA rank r=8, α=16, din=4096, dou=4096, and maximum=1024.

[0071] Training process: Based on 6 datasets of highly agreeable traits, 6 LoRA expert modules (M_gratitude, M_tolerance, M_generosity, M_kindness, M_compassion, M_trust) were trained respectively. Each model was trained for 3 rounds with a batch size of 2. The AdamW optimizer was used with a learning rate of 1e-4.

[0072] Training results: The parameter size of each LoRA expert module is approximately 80MB, which is only 0.0008192% of the original model parameters.

[0073] S104. Combine multiple different personality sub-traits and train the optimal weights of the personality sub-trait LoRA expert module on the corresponding personality trait dataset, thereby achieving controllable editing and precise adjustment of the personality traits loaded by the LLaMA-3-8B-Instruct model. Personality trait dataset construction: 500 samples were randomly selected from the training datasets of each of the 6 LoRA expert modules to synthesize a dataset of 3000 highly agreeable personality traits for gating function training.

[0074] Gated function training: The gated network adopts a 2-layer MLP structure with 256 hidden layers, ReLU activation function, and an output dimension of "6×224" (224 layers after applying LoRA to the LLaMA-3-8B-Instruct model, 6 sub-features × 224 LoRA layers). The training is conducted by minimizing the loss function. Training gating parameters The iterations were performed in 3 rounds, using the AdamW optimizer with an initial learning rate of 1e-4.

[0075] Model generation: The six trained LoRA expert modules are fused with a gating function to obtain a highly spontaneous model M. A+ Its forward propagation process is as follows: .

[0076] Effect Verification: Input scenario question: "If a friend accidentally breaks your valuable item and apologizes, how would you respond?", M A+ The generated response, “It’s okay, I know you didn’t mean it. The item is precious, but our friendship is more important. Just be a little more careful next time you use it. Don’t feel too guilty,” was reviewed by experts and found to fully meet the six sub-trait requirements of high agreeableness, demonstrating a consistent and natural personality expression.

[0077] When deployed in a virtual customer service scenario to handle user complaints and inquiries, the user satisfaction rate reached 92%, which is 35% higher than the complaint resolution rate of the traditional customer service model. When deployed in a game NPC scenario, as a friendly character interacting with players, the average interaction time of players was extended by 40%, and the interaction experience score reached 4.7 / 5.0.

[0078] In this invention: The Five-Factor Model is a widely used personality theory model that divides personality traits into five dimensions: Openness (O), Conscientiousness (C), Extraversion (E), Agreeableness (A), and Neuroticism (N). It has become one of the mainstream models for describing individual personality traits.

[0079] LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method that introduces a low-rank decomposition layer onto the weight matrix of a pre-trained model, allowing the model to be adapted to a specific task by updating only a small number of new parameters.

[0080] MoE (Mixture of Experts) is a model structure that allows multiple "expert" sub-models to exist in parallel, and a gating network dynamically selects or weights some experts to participate in reasoning, thereby improving model performance and computational efficiency.

[0081] MLP, short for Multi-Layer Perceptron, is a basic artificial neural network consisting of an input layer, hidden layers, and an output layer. It is primarily used for supervised learning tasks. It is widely applied to tasks such as classification and regression, and is one of the commonly used models for handling complex problems.

[0082] LLaMA-3-8B-Instruct is a large language model focused on dialogue optimization and fine-tuning. It employs an autoregressive architecture consisting of 32 Transformer blocks and uses a multi-head attention mechanism to capture complex relationships between texts in parallel. It leverages the SwiGLU activation function and RMSnorm layer normalization to optimize performance, combining instruction fine-tuning and quantization adaptation capabilities to balance hardware overhead while ensuring smooth and accurate human-computer interaction.

[0083] Expert Background: All experts involved in this invention hold doctoral degrees in psychology and hold the positions of professor or associate professor.

[0084] Comparison with existing technologies: The results, based on the Big Five Personality Inventory applicable to large language models, show that the present invention improves upon the optimal prompting method by 22% and significantly improves upon the existing LoRA fine-tuning method by 84%.

[0085] The embodiments of the present invention have been described in detail above, but the content described is only a preferred embodiment of the present invention and should not be considered as limiting the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the present invention.

Claims

1. A method for fine-grained editing of personality traits in a large language model, characterized in that: Includes the following steps: S101. Obtain a question-and-answer dataset containing personality trait-themed scenarios for subsequent model training; S102. Perform multiple rounds of backtracking verification and quality optimization on the question-and-answer dataset to construct a personality sub-trait question-and-answer dataset; S103. Based on the constructed personality subtrait question-and-answer dataset, the LoRA efficient supervised fine-tuning method is used to train multiple LoRA expert modules for specific personality subtraits. S104. Multiple LoRA expert modules with different personality sub-traits are fused based on the MoE framework, and a gating function is trained on the corresponding personality trait question-and-answer dataset to learn the optimal combination weights for dynamically allocating personality sub-trait LoRA expert modules, thereby achieving controllable editing and precise adjustment of personality traits loaded from the LLaMA-3-8B-Instruct model.

2. The method for implementing fine-grained editing of personality traits in large language models according to claim 1, characterized in that: In step S101, the question-and-answer dataset is generated in the following way: using the Big Five personality trait descriptions as prompt information, and calling the GPT-4o model to generate corresponding responses for each situational question in the question-and-answer dataset. The generated responses must conform to the expected personality trait settings.

3. The method for implementing fine-grained editing of personality traits in large language models according to claim 2, characterized in that: In step S101, the Big Five personality traits include positive personality traits and negative personality traits. Both positive and negative personality traits include five dimensions: openness, conscientiousness, extraversion, agreeableness, and neuroticism.

4. The method for implementing fine-grained editing of personality traits in large language models according to claim 2, characterized in that: In step S101, the prompt word format for the GPT-4o response generation is: system instruction + personality trait description + output requirement + scenario question.

5. The method for implementing fine-grained editing of personality traits in large language models according to claim 1, characterized in that: In step S102, the multi-round backtracking verification and quality optimization includes: using DeepSeek-V3 to score all responses generated by GPT-4o from 1 to 5 points according to their fit with the expected sub-features. A score of 4 or above indicates that the data quality is high. If the score is below 4 points, it is considered that the sub-feature label does not match the sub-feature captured by the model, and the data will be returned to GPT-4o for response improvement. If the same data has a score below 4 points in three consecutive iterations, it will be discarded directly.

6. The method for implementing fine-grained editing of personality traits in a large language model according to claim 5, characterized in that: In step S102, the filtered data is sampled and reviewed by human experts. 10% of the samples are drawn from each sub-trait category. If three or more experts determine that the data is qualified, the data is valid. If the qualification rate of a sub-trait category is less than 90%, the generation process is optimized and the data is reviewed again.

7. The method for implementing fine-grained editing of personality traits in large language models according to claim 1, characterized in that: In step S103, the LoRA efficient supervised fine-tuning method is implemented by freezing the basic weights of the original model LLaMA-3-8B-Instruct. Restricting parameter updates to a low-rank decomposition form, this process can be represented as: ； Where ΔW0 represents the updated parameters during the training phase. and Let din and dou represent two low-rank matrices, where din and dou represent the dimensions of the input and output features, respectively, and their ranks are 1. Training is performed only on the parameters in the two low-rank matrices A and B; Based on the aforementioned personality subtrait question-and-answer dataset, 2m×n LoRA expert modules are trained. Each LoRA expert module is dedicated to learning and expressing a specific personality subtrait, where m represents the m personality traits corresponding to one personality trait, and n represents the n personality traits to be controlled by the model.

8. The method for implementing fine-grained editing of personality traits in large language models according to claim 1, characterized in that: In step S104, the specific implementation of the trait fusion includes: aggregating the LoRA expert modules of each sub-trait into the LLaMA-3-8B-Instruct model in a MoE manner, and introducing a trainable weight parameter ω into each LoRA layer, such that the input of the layer is multiplied by the weight parameter ω before entering the corresponding LoRA expert module, and treating the LoRA expert module of each sub-trait as an independent expert. Its weight matrix It can be represented as: ； in, and Let din and dou represent two low-rank matrices, where din and dou represent the dimensions of the input and output features, respectively, and their ranks are 1. The gating function is responsible for learning and assigning the optimal combination of weights to simulate the target personality, for the Big Five personality scenarios. Where O represents openness; C represents conscientiousness; E represents extraversion; A represents agreeableness; and N represents neuroticism, the model simulates personality types. The forward propagation process of time can be represented as: ； Among them, O pi The model represents the simulated personality P. i Output at time; W 0x and ∆W Ex These represent the base weights of the original model LLaMA-3-8B-Instruct and the LoRA expert module-specific weights, respectively; M represents the total number of LoRA expert modules used to simulate sub-personality traits in the current MoE architecture; x represents the input vector and the gating function. It is a multilayer perceptron network with 256 hidden layers and ReLU activation function. G This represents the parameter matrix used for learning in the gating mechanism; the input dimension of the gating network is the size of the model's hidden layers, and the output dimension is the number of personality sub-traits × the number of LoRA layers; the gating function... Dynamically adjust the output weights based on the input x To achieve personality simulation by scaling the input of the LoRA expert module for the j-th sub-trait, B j A j Let represent the LoRA low-rank weight matrix corresponding to the j-th expert module, where α and r are the constant scaling factor and the rank parameter of the LoRA expert module, respectively.

Images