Personality digital person construction method and system based on prototype person multi-modal data
By extracting and quantifying the personality genes of the prototype, a digital human capable of perceiving, thinking, and expressing himself like the prototype was constructed. This solves the problem that digital humans cannot simulate personality in existing technologies and achieves highly consistent digital human construction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING ZHILUN ROTATION TECHNOLOGY CO LTD
- Filing Date
- 2026-01-26
- Publication Date
- 2026-05-12
AI Technical Summary
Existing technologies cannot effectively simulate the personality of the prototype when creating digital humans, causing the digital humans to expose their inhuman nature in continuous interaction and failing to provide deep emotional connection and personal trust.
By acquiring multimodal data from the prototype, preprocessing it, and inputting it into a large-scale language model and sentiment computing model, we can extract language style, emotion-cognitive association, and value decision-making features, integrate them into a personality gene vector, and perform consistency verification and iterative optimization in the AI model to construct a digital human that can perceive, think, and express itself like the prototype.
It has achieved a high degree of consistency between the digital human and the prototype at the personality level, with continuous emotional connection and personality trust, which is a technological breakthrough that transcends the resemblance in form to the likeness in spirit, and can be applied to fields such as digital immortality, emotional companionship and personalized inheritance.
Smart Images
Figure CN122020533A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, specifically to a method and system for constructing a personality digital human based on prototype multimodal data. Background Technology
[0002] With the rapid development of technologies such as artificial intelligence, computer graphics, natural language processing, and speech synthesis, digital humans, as an important carrier for the integration of virtual and reality, are gradually expanding from film and television special effects and gaming entertainment to multiple industry application scenarios such as education, healthcare, finance, and customer service. Digital humans refer to virtual characters constructed through digital technology that possess human appearance, behavior, and even emotional interaction capabilities. Their core technologies encompass 3D modeling, motion capture, voice-driven computing, affective computing, and large-scale model-driven intelligent dialogue systems. In recent years, thanks to breakthroughs in deep learning and generative AI, the realism, interactivity, and intelligence of digital humans have significantly improved, making them one of the key entry points for a new paradigm of human-computer interaction.
[0003] Existing technological solutions for creating digital humans primarily focus on 3D modeling of appearance, cloning of voice timbre, or imitation of simple conversational habits. This replication is superficial, fragmented, and inconsistent. The result is often that while the digital human may occasionally say or make a certain expression that resembles the original, in continuous interaction, its internal logic, emotional response patterns, and values will quickly expose its "inhuman" nature, failing to provide deep emotional connection and personal trust. Summary of the Invention
[0004] Therefore, this application provides a method and system for constructing a digital personality based on multimodal data of a prototype person, in order to solve the problem that digital humans in the existing technology cannot simulate the personality of the prototype person.
[0005] To achieve the above objectives, this application provides the following technical solution:
[0006] Firstly, a method for constructing a digital personality based on multimodal data of a prototype individual includes:
[0007] Step 1: Obtain raw personality data that reflects the prototype's personality from the prototype's language data, voice data, and behavioral data;
[0008] Step 2: Preprocess the raw personality data;
[0009] Step 3: Input the preprocessed original personality data into a large language model for deep embedding representation to obtain language style fingerprint features;
[0010] Step 4: Input the preprocessed raw personality data into the emotion computing model to analyze the prototype's emotional expression, and associate the emotional expression with cognitive roots to obtain emotion-cognitive association features;
[0011] Step 5: Extract the values from the preprocessed raw personality data using topic modeling and causal inference, and simulate the value ranking and selection tendencies of the prototype person under specific dilemmas through decision preference tree to obtain value decision characteristics;
[0012] Step 6: Fuse the language style fingerprint features, the emotion-cognition association features, and the value decision features to obtain the personality gene vector;
[0013] Step 7: Input the personality gene vector into the pre-built AI model for personality replication, and perform consistency verification and iterative optimization to obtain the final digital personality.
[0014] Preferably, step 2, when preprocessing the original personality data, includes: desensitizing, cleaning, removing noise and invalid information from the original personality data, and aligning the timestamps.
[0015] Preferably, in step 7, when the personality gene vector is input into the pre-built AI model for personality replication, the AI model uses a pre-trained basic dialogue model; the basic dialogue model needs to be fine-tuned for personality conditionalization.
[0016] Preferably, the personality conditional fine-tuning employs P-tuning or LoRA fine-tuning techniques.
[0017] Preferably, the basic dialogue model adopts a large language model based on the Transformer architecture.
[0018] Preferably, in step 7, when the personality gene vector is input into a pre-built AI model for personality replication, the AI model uses the pre-model as the execution engine.
[0019] Preferably, step 7 includes objective consistency verification and subjective Turing verification.
[0020] Secondly, a personality digital human construction system based on prototype multimodal data includes:
[0021] The personality data acquisition module is used to obtain raw personality data that reflects the prototype's personality from the prototype's language data, voice data, and behavioral data.
[0022] The data preprocessing module is used to preprocess the raw personality data;
[0023] The language style fingerprint feature extraction module is used to input the preprocessed original personality data into a large language model for deep embedding representation to obtain language style fingerprint features;
[0024] The emotion-cognitive association feature extraction module is used to input the preprocessed original personality data into the emotion computing model to analyze the emotional expression of the prototype, and associate the emotional expression with the cognitive root to obtain the emotion-cognitive association features;
[0025] The value decision feature extraction module is used to extract the values of the preprocessed original personality data using topic modeling and causal inference, and to simulate the value ranking and selection tendencies of the prototype person under specific dilemmas by using a decision preference tree to obtain value decision features.
[0026] The feature fusion module is used to fuse the language style fingerprint features, the emotion-cognition association features, and the value decision features to obtain a personality gene vector;
[0027] The personality digital human construction module is used to input the personality gene vector into a pre-built AI model for personality replication, and to perform consistency verification and iterative optimization to obtain the final personality digital human.
[0028] Thirdly, a computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method for constructing a personality digital human based on prototype multimodal data.
[0029] Fourthly, a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method for constructing a personality digital human based on prototype multimodal data.
[0030] Compared with the prior art, this application has at least the following beneficial effects:
[0031] This application provides a method for constructing a digital personality based on multimodal data of a prototype. It involves acquiring and preprocessing the prototype's raw personality data; extracting language style fingerprint features, emotion-cognitive association features, and value decision-making features from the preprocessed raw personality data, and fusing them to obtain a personality gene vector; inputting the personality gene vector into a pre-constructed AI model for personality replication, and performing consistency verification and iterative optimization to obtain the final digital personality. This application extracts and quantifies the prototype's personality genes, then internalizes these genes into the AI model's inherent logical driving force, thereby constructing a digital life form capable of perceiving, thinking, making decisions, and expressing itself like the prototype (i.e., capable of simulating the prototype's personality), achieving a leap from mere "formal resemblance" to "spiritual resemblance." Attached Figure Description
[0032] To more intuitively illustrate the prior art and this application, exemplary drawings are provided below. It should be understood that the specific shapes and structures shown in the drawings should not generally be regarded as limiting conditions for implementing this application; for example, based on the technical concept disclosed in this application and the exemplary drawings, those skilled in the art are able to easily make conventional adjustments or further optimizations to the addition / reduction / classification, specific shapes, positional relationships, connection methods, size ratios, etc. of certain units (components).
[0033] Figure 1 This is a flowchart illustrating a method for constructing a digital personality based on multimodal data of a prototype person, as provided in Embodiment 1 of this application. Detailed Implementation
[0034] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0035] In the description of this application: unless otherwise stated, "a plurality of" means two or more. The terms "first," "second," "third," etc., in this application are intended to distinguish the objects referred to and do not have any special meaning in terms of technical connotation (e.g., they should not be construed as an emphasis on importance or order). Expressions such as "including," "comprising," and "having" also mean "not limited to" (certain units, components, materials, steps, etc.).
[0036] The terms used in this application, such as "upper," "lower," "left," "right," and "middle," are generally used to indicate the general relative positional relationship for the purpose of intuitive understanding by referring to the accompanying drawings, and are not absolute limitations on the positional relationship in the actual product.
[0037] Example 1
[0038] Please see Figure 1 This application provides a method for constructing a personality digital human based on multimodal data of a prototype person. The aim is to construct and reproduce a virtual AI digital human representing an individual's intrinsic personality through a deep learning model. The method includes:
[0039] S1: Obtain raw personality data that reflects the prototype's personality from the prototype's language data, voice data, and behavioral data;
[0040] Specifically, language data includes: audio and video transcriptions of personal communication processes, chat logs, social media posts, emails, diaries, and works (articles, code, and design drafts), etc.
[0041] The audio data includes long audio recordings of individuals in various emotional states (happiness, sadness, anger, and calmness).
[0042] Behavioral data (optional) includes: videos of an individual's historical communications, which can be analyzed to identify nonverbal behavioral characteristics such as micro-expressions, embodied language, and reaction delays in different social scenarios.
[0043] This step can comprehensively and unbiasedly capture the original data of the "prototype" person that reflects their personality from the three data sources mentioned above, namely the original personality data, which is multi-source data.
[0044] S2: Preprocess the raw personality data;
[0045] Specifically, this step of preprocessing the raw personality data includes: desensitizing, cleaning, removing noise and invalid information from the raw personality data, and aligning the timestamps of the multi-source data to lay a high-quality data foundation for subsequent in-depth analysis.
[0046] S3: Input the preprocessed raw personality data into a large language model for deep embedding representation to obtain language style fingerprint features;
[0047] Specifically, this step uses a large language model to perform deep embedding representation on the preprocessed raw personality data. This not only extracts "lexical richness", "sentence pattern preference" and "emotional tendency", but also extracts implicit style features that are difficult to define explicitly, such as humor, irony patterns, argumentation style and narrative rhythm, through contrastive learning, adversarial generative networks and other techniques, thus obtaining language style fingerprint features.
[0048] S4: Input the preprocessed raw personality data into the emotion computing model to analyze the emotional expression of the prototype, and associate the emotional expression with the cognitive roots to obtain the emotion-cognitive association features;
[0049] Specifically, this step utilizes an affective computing model to analyze the emotional expression of preprocessed raw personality data on different issues (triggers) and correlates it with the underlying cognitive roots, thereby obtaining affective-cognitive correlation features. For example, a multivariate mapping model is constructed that "when discussing 'technology ethics,' the emotional tendency is 'anxiety,' and the cognitive pattern is 'critical thinking.'" This multivariate mapping model goes beyond simple "emotional classification" and constructs a "stimulus-cognition-emotion" response chain.
[0050] S5: Use topic modeling and causal inference to extract values from preprocessed raw personality data, and simulate the value ranking and selection tendencies of prototype individuals under specific dilemmas through decision preference trees to obtain value decision characteristics.
[0051] Specifically, this step analyzes the preprocessed raw personality data (including the prototype's statements, works, and behaviors), using topical models and causal inference to extract its core values, worldview, and principles. More importantly, this step also uses a constructed decision preference tree to simulate its value ranking and selection tendencies under specific dilemmas, thereby obtaining value decision characteristics.
[0052] S6: The language style fingerprint features, emotion-cognitive association features, and value decision features are fused to obtain the personality gene vector;
[0053] Specifically, this step integrates and reduces the high-dimensional features such as language style fingerprint features, emotion-cognitive association features, and value decision features extracted in steps S3-S5 through an end-to-end neural network, and finally outputs a high-dimensional, dense "personality gene vector", which summarizes the core personality information of the prototype.
[0054] In this embodiment, steps S3-S6 do not involve simple psychological scale scoring. Instead, they involve constructing a computable and generative "Personality Gene Vector" from massive amounts of unstructured data through AI model self-learning and extraction. This Personality Gene Vector is a mathematical expression of the prototype personality.
[0055] S7: Input the personality gene vector into a pre-built AI model to replicate the personality, and perform consistency verification and iterative optimization to obtain the final digital personality.
[0056] Specifically, the purpose of this step is to inject an abstract "personality gene vector" into the AI model, so that the content it generates naturally carries the characteristics of the prototype. To achieve this goal, this embodiment proposes two parallel technical paths, including:
[0057] Option A: Personality Conditional Fine-tuning (Deep Integration)
[0058] Model selection: Choose a powerful, pre-trained base dialogue model (such as an LLM based on the Transformer architecture).
[0059] Personality-Conditioned Fine-Tuning: This approach directly "imprints" personality gene vectors into the model's parameters. Employing efficient fine-tuning techniques such as P-tuning and LoRA, the "personality gene vectors" serve as learnable cues or adapter weights, locked onto specific layers of the model. During fine-tuning, the model learns not just scattered knowledge points, but the global logic of "how a certain personality should think and organize language," making personality an intrinsic and stable constraint on the model.
[0060] Multimodal synchronization: When outputting voice, facial expressions, etc., the "personality gene vector" is used as an additional input condition to ensure that the tone and expression are consistent with the inner personality.
[0061] Option B: A smart agent framework without fine-tuning (externally driven)
[0062] Model selection: Use powerful, pre-built models that do not require fine-tuning as the "execution engine".
[0063] Personality Vector Intelligentization: This approach does not change the model itself, but treats personality as an "external soul." Through a lightweight personality parser, the "personality gene vector" is decoded in real time into specific language styles, values, and emotional descriptions.
[0064] Dynamic prompt construction: In each conversation, the intelligent system dynamically constructs a structured prompt by combining the user's question, the decoded personality description, and the conversational context, and then sends it to the "execution engine" for response. This is equivalent to equipping the model with a "personality manual," guiding it to react in accordance with the prototype personality in each interaction.
[0065] Generate Validation: After the model generates the response, an evaluation module can perform a consistency check based on the "personality description" to ensure that the output does not deviate from the core traits of the prototype.
[0066] Option C: A personality-conditional fine-tuning agent framework (i.e., a combination of Option A and Option B).
[0067] Specifically, this embodiment can also construct a feedback loop to continuously optimize the model by quantitatively evaluating the replication effect of the two models mentioned above, including:
[0068] Objective consistency verification: Design a test set (“thought experiment” set) containing a large number of situational traps and value conflicts to automatically test whether the digital human’s reactions are highly consistent with the prediction results of the prototype’s “personality gene vector”.
[0069] Subjective Turing Verification (Enhanced Version): Invite multiple people who are very close to the prototype to conduct multiple rounds of open-ended dialogues, and conduct structured scoring and qualitative descriptions from multiple dimensions such as "personality similarity", "emotional resonance", "logical consistency" and "indistinguishability".
[0070] Feedback-driven iteration: The "distortion points" found in the verification process are used as negative samples, along with the original data, to carry out a new round of incremental fine-tuning of the feature extraction model or AI model of personality gene vector, forming a closed loop of continuous optimization.
[0071] This embodiment provides a method for constructing a digital personality based on multimodal data of a prototype. By extracting and quantifying the "personality genes" of the prototype and internalizing them into the internal logical driving force of the AI model, a digital life form capable of perceiving, thinking, making decisions, and expressing itself like the real person is ultimately constructed. That is, the digital life form is highly consistent with its "prototype" at the personality level, achieving a leap from "formal resemblance" to "spiritual resemblance," enabling it to realize unprecedented application value in fields such as digital immortality, emotional companionship, personalized heritage inheritance, and role simulation consultants.
[0072] The method for constructing a digital personality based on multimodal data of a prototype person provided in this embodiment is fundamentally different from existing technologies, as shown in Table 1:
[0073] Table 1
[0074]
[0075] Example 2
[0076] This embodiment provides a system for constructing a digital personality based on multimodal data of a prototype person, including:
[0077] The personality data acquisition module is used to obtain raw personality data that reflects the prototype's personality from the prototype's language data, voice data, and behavioral data.
[0078] The data preprocessing module is used to preprocess the raw personality data;
[0079] The language style fingerprint feature extraction module is used to input the preprocessed original personality data into a large language model for deep embedding representation to obtain language style fingerprint features;
[0080] The emotion-cognitive association feature extraction module is used to input the preprocessed original personality data into the emotion computing model to analyze the emotional expression of the prototype, and associate the emotional expression with the cognitive root to obtain the emotion-cognitive association features;
[0081] The value decision feature extraction module is used to extract the values of the preprocessed original personality data using topic modeling and causal inference, and to simulate the value ranking and selection tendencies of the prototype person under specific dilemmas by using a decision preference tree to obtain value decision features.
[0082] The feature fusion module is used to fuse the language style fingerprint features, the emotion-cognition association features, and the value decision features to obtain a personality gene vector;
[0083] The personality digital human construction module is used to input the personality gene vector into a pre-built AI model for personality replication, and to perform consistency verification and iterative optimization to obtain the final personality digital human.
[0084] For details on the specific implementation of each module in a personality digital human construction system based on prototype multimodal data, please refer to the above description of the limitations of a personality digital human construction method based on prototype multimodal data, which will not be repeated here.
[0085] Example 3
[0086] This embodiment provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method for constructing a personality digital human based on prototype multimodal data.
[0087] Example 4
[0088] This embodiment provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method for constructing a personality digital human based on prototype multimodal data.
[0089] The technical features of the above embodiments can be combined in any way (as long as there is no contradiction in the combination of these technical features). For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described; these embodiments not explicitly written should also be considered to be within the scope of this specification.
Claims
1. A method for constructing a digital personality based on multimodal data of a prototype person, characterized in that, include: Step 1: Obtain raw personality data that reflects the prototype's personality from the prototype's language data, voice data, and behavioral data; Step 2: Preprocess the raw personality data; Step 3: Input the preprocessed original personality data into a large language model for deep embedding representation to obtain language style fingerprint features; Step 4: Input the preprocessed raw personality data into the emotion computing model to analyze the prototype's emotional expression, and associate the emotional expression with cognitive roots to obtain emotion-cognitive association features; Step 5: Extract the values from the preprocessed raw personality data using topic modeling and causal inference, and simulate the value ranking and selection tendencies of the prototype person under specific dilemmas through decision preference tree to obtain value decision characteristics; Step 6: Fuse the language style fingerprint features, the emotion-cognition association features, and the value decision features to obtain the personality gene vector; Step 7: Input the personality gene vector into the pre-built AI model for personality replication, and perform consistency verification and iterative optimization to obtain the final digital personality.
2. The method for constructing a digital personality based on multimodal data of a prototype person according to claim 1, characterized in that, Step 2, when preprocessing the original personality data, includes: desensitizing, cleaning, removing noise and invalid information from the original personality data, and aligning the timestamps.
3. The method for constructing a digital personality based on multimodal data of a prototype person according to claim 1, characterized in that, In step 7, when the personality gene vector is input into the pre-built AI model for personality replication, the AI model uses a pre-trained basic dialogue model; the basic dialogue model needs to be fine-tuned for personality conditionalization.
4. The method for constructing a digital personality based on multimodal data of a prototype person according to claim 3, characterized in that, The personality conditional fine-tuning uses P-tuning or LoRA fine-tuning techniques.
5. The method for constructing a digital personality based on multimodal data of a prototype person according to claim 3, characterized in that, The basic dialogue model adopts a large language model based on the Transformer architecture.
6. The method for constructing a digital personality based on multimodal data of a prototype person according to claim 1, characterized in that, In step 7, when the personality gene vector is input into the pre-built AI model for personality replication, the AI model uses the pre-model as the execution engine.
7. The method for constructing a digital personality based on multimodal data of a prototype person according to claim 1, characterized in that, In step 7, the consistency verification includes objective consistency verification and subjective Turing verification.
8. A system for constructing a digital personality based on multimodal data of a prototype person, characterized in that, include: The personality data acquisition module is used to obtain raw personality data that reflects the prototype's personality from the prototype's language data, voice data, and behavioral data. The data preprocessing module is used to preprocess the raw personality data; The language style fingerprint feature extraction module is used to input the preprocessed original personality data into a large language model for deep embedding representation to obtain language style fingerprint features; The emotion-cognitive association feature extraction module is used to input the preprocessed original personality data into the emotion computing model to analyze the emotional expression of the prototype, and associate the emotional expression with the cognitive root to obtain the emotion-cognitive association features; The value decision feature extraction module is used to extract the values of the preprocessed original personality data using topic modeling and causal inference, and to simulate the value ranking and selection tendencies of the prototype person under specific dilemmas by using a decision preference tree to obtain value decision features. The feature fusion module is used to fuse the language style fingerprint features, the emotion-cognition association features, and the value decision features to obtain a personality gene vector; The personality digital human construction module is used to input the personality gene vector into a pre-built AI model for personality replication, and to perform consistency verification and iterative optimization to obtain the final personality digital human.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.