A method for generating a digital human

Abstract

The application discloses a digital person generation method, comprising: inputting text content, preprocessing the text content, segmenting the text content into single words or vocabularies, and extracting keywords in the text; performing sentiment polarity labeling according to the text content to determine the sentiment tendency of the text; converting the text content into audio and generating a timestamp; generating a keyword time sequence according to the keywords extracted in step S101 combined with the timestamp; and generating a digital person according to the word segmentation result, sentiment tendency, audio conversion and timestamp information obtained in steps S101 to S104, and the keyword time sequence. By integrating the successfully matched action sequence into the action chain, the coherence and smooth integration of discrete action sequences are realized, and the action of the digital person is more natural and real. The use of the action chain enhances the coherence and fluency between multiple actions of the digital person, and the action transition is smoother.

Description

Technical Field

[0001] This invention relates to the field of digital human technology, and more particularly to a method for generating digital humans. Background Technology

[0002] With the rapid development of artificial intelligence technology, digital humans, as an emerging form of digital media, have shown enormous application potential in various fields such as entertainment, education, and customer service. Digital humans not only possess high customizability and interactivity, but can also provide a more natural and immersive user experience by simulating the expressions, movements, and voices of real humans. However, generating a vivid and realistic digital human requires comprehensive consideration of technologies such as text processing, sentiment analysis, audio conversion, and motion generation.

[0003] For example, Chinese Patent Publication No. CN117808942B discloses a method and system for generating semantically strongly correlated 3D digital human actions. The method includes: preprocessing the input text to obtain the corresponding audio, keyword sequence, and word timestamp of each word in the text; generating a base action sequence corresponding to the audio based on a base action generation model; searching and matching the keywords in the keyword sequence with the semantics in the action sequence database to calculate the action sequence corresponding to each keyword and obtain a keyword-action dictionary; and fusing the word timestamp of each word and the keyword-action dictionary into the base action sequence in chronological order to output a semantically strongly correlated 3D digital human action sequence. The 3D digital human actions generated by this invention are more natural and vivid, and meet the needs of real-time interaction between the 3D digital human and the user.

[0004] A significant flaw in the comparison file is the lack of necessary correction and optimization regarding the continuity between two actions performed by the digital human. This oversight leads to a noticeable problem: when the digital human performs a series of actions, the transitions between them appear abrupt and unnatural. When the digital human shifts from one action to another, the lack of smooth transitions makes the gaps and abruptness in the movements clearly perceptible to the viewer. This disjointed performance not only disrupts the overall fluidity of the digital human's movements but also significantly reduces viewer immersion and satisfaction. To address this issue, meticulous adjustments and optimizations to the digital human's movements are necessary. Summary of the Invention

[0005] This application provides a method for generating digital humans by inserting successfully matched action sequences into action chains, integrating discrete action sequences into a coherent and smooth action chain, making the digital human's action performance more natural and realistic.

[0006] This application provides a method for generating a digital human, including: S101, Input text content, preprocess the text content, segment the text content into individual words or words, and extract keywords from the text; S102, Based on the text content, perform sentiment polarity labeling to determine the sentiment tendency of the text; S103, convert the text content into audio and generate a timestamp; S104. Based on the keywords extracted in step S101, and combined with the timestamps, generate a keyword time series. S105. Based on the word segmentation results, sentiment trends, audio conversion and timestamp information obtained in steps S101 to S104, and the keyword time series, generate a digital human.

[0007] Preferably, based on the extracted keywords, an empty keyword sequence list is created to store keywords with timestamps. The extracted keyword list is traversed, and for each keyword, it is combined with timestamp information, which includes start and end times.

[0008] Preferably, the steps for combining the extracted keywords into keyword blocks, and the action chain composed of keyword blocks and action sequences are as follows: S201, Based on the word segmentation results of step S101, combine the keywords into keyword blocks; Based on a number of words or words, the co-occurrence frequency of each word with other words in the text is counted, and words with a co-occurrence frequency that reaches a preset frequency are combined into keyword blocks. S202, Build an action sequence database and store it in the form of key-value pairs; S203, use an algorithm to semantically match keyword blocks with an action sequence database; S204 inserts the successfully matched action sequence into the action chain according to the order in which the keyword blocks appear in the text.

[0009] Preferably, the keyword block is semantically matched with the action sequence database. If the similarity exceeds a preset threshold, the keyword block is successfully matched with the semantics, and the action sequence corresponding to the semantics is matched. If the keyword block cannot find a matching semantic in the database, the final action sequence is determined according to preset rules.

[0010] Preferably, based on the keyword time sequence generated in step S104, the order in which the keyword blocks appear in the text is determined, and the corresponding action sequences are inserted into the action chain in sequence according to the order of the keyword blocks. The action chain consists of multiple consecutive action sequences, each of which represents a series of consecutive actions of the digital human.

[0011] Preferably, keyword blocks are matched with an action sequence database. The degree of matching between keyword blocks and action sequences is adjusted by a matching factor. The matching factor is determined by influencing factors, including the importance, emotional intensity, frequency of occurrence, and position of the keyword block in the text.

[0012] Preferably, the importance is set based on the role of the keyword block in the text, the sentiment intensity is set based on the result of the sentiment analysis algorithm, the frequency of occurrence is set based on the number of times the keyword block appears in the text, and the position is set based on the position of the keyword block in the paragraph or sentence of the text.

[0013] Preferably, the matching factor = importance weight (importance value) + emotional intensity weight (emotional intensity value).

[0014] Preferably, the sentiment polarity of the text content is labeled based on a dictionary method. Each word in the text content is traversed and its corresponding polarity in the sentiment dictionary is found. Then, the sentiment score of the entire text is calculated based on the polarity and weight of the words. Finally, the sentiment tendency of the text is determined based on the score.

[0015] One or more technical solutions provided in this application have at least the following technical effects or advantages: By employing a word segmentation technology that combines dictionary analysis and statistical methods, keywords in text can be accurately and comprehensively identified, covering new vocabulary and technical terms, significantly improving the efficiency and accuracy of text processing. Sentiment polarity annotation using a dictionary can quickly grasp the emotional direction of the text, aiding in a deeper understanding of the text content and providing emotional support for digital human interaction. By integrating information such as word segmentation, sentiment analysis, audio conversion, timestamps, and keyword time series, the created digital human possesses a rich interactive experience, better meeting user needs and improving user satisfaction. In terms of motion processing, by integrating successfully matched motion sequences into motion chains, the coherence and smooth integration of discrete motion sequences are achieved, making the digital human's movements more natural and realistic. The use of motion chains enhances the coherence and smoothness between multiple movements of the digital human, making motion transitions smoother; Furthermore, by comprehensively considering multiple factors such as the importance, emotional intensity, frequency of occurrence, and position of keyword blocks in the text, and quantifying these factors, the matching of keyword blocks with action sequences can be achieved more accurately. Based on the values ​​of the matching factors, parameters such as the weight, amplitude, and duration of the action sequence are dynamically adjusted, making key nodes in the action chain more prominent and non-key nodes relatively less prominent, thereby enhancing the overall expressiveness and communication effect. Attached Figure Description

[0016] Figure 1This is a flowchart illustrating a method for generating a digital human according to the present invention. Figure 2 This is a flowchart illustrating the action chain composed of keyword blocks and action sequences in an embodiment of the present invention. Detailed Implementation

[0017] To facilitate understanding of the present invention, a more complete description of this application will be given below with reference to the accompanying drawings, which illustrate preferred embodiments of the invention. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to enable a more thorough and complete understanding of the disclosure of the present invention.

[0018] It should be noted that the terms "vertical," "horizontal," "up," "down," "left," "right," and similar expressions used in this article are for illustrative purposes only and do not represent the only possible implementation.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention; the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0020] Example 1: Figure 1 This is a flowchart illustrating a method for generating a digital human according to an embodiment of the present invention, including the following steps: S101, Input text content, preprocess the text content, segment the text content into individual words or words, and extract keywords from the text; Specifically, punctuation marks, spaces, and special characters are removed from the text. Based on a dictionary containing specialized vocabulary, the dictionary is loaded into the word segmentation tool jieba to segment the text. Jieba segmentation uses the vocabulary in the dictionary and its built-in segmentation algorithm to cut the text into words or phrases. After initial segmentation, the results are iterated, and text content not matched by the dictionary is further processed using statistical segmentation. A Hidden Markov Model is trained to segment text content not matched by the dictionary, converting it into character sequences and marking known word boundaries. These marked word boundaries are then used to further refine the segmentation process. A Hidden Markov Model (HMM) is trained to learn the mapping probability from characters to word boundaries. Based on the co-occurrence frequency of words in the text and contextual information, potential word boundaries are identified. The dictionary-based and statistical-based word segmentation results are merged to obtain a complete word segmentation result set, which includes both known words from the dictionary and new words and technical terms identified through statistical methods. The segmented words and vocabulary are organized into a vocabulary list, and the frequency of each word is counted to form a vector. Each dimension of the vector corresponds to a word in the vocabulary list, and the value of the vector is the frequency of that word in the document. Words with high vector values ​​are extracted.

[0021] S102, perform sentiment polarity labeling based on text content to determine the sentiment tendency of the text.

[0022] The sentiment polarity of the text is labeled using a dictionary-based approach. Each word in the text is traversed and its corresponding polarity in the sentiment dictionary is found. Then, the sentiment score of the entire text is calculated based on the polarity and weight of the words. Finally, the sentiment tendency of the text is determined based on the score.

[0023] S103 converts the text content into audio and generates a timestamp.

[0024] This patent utilizes a Text-to-Speech (TTS) algorithm to convert text into speech. Parameters are configured, including speech model, speech rate, pitch, and volume, based on the selected TTS engine. The text is input into the TTS engine, which then converts it into an audio file according to the configured parameters. The TTS algorithm is existing technology and will not be described in detail here. The TTS engine's timestamp generation function identifies the speech content within the audio text, converting the audio signal into a text representation. The text is then segmented into individual characters or words, and a timestamp is assigned to each character or word. This timestamp typically includes a start time and an end time, indicating the position of the character or word in the audio. The generated timestamp information is then saved.

[0025] S104. Based on the keywords extracted in step S101, and combined with the timestamps, generate a keyword time series.

[0026] Based on the keywords, create an empty keyword sequence list to store the timestamped keywords. Iterate through the extracted keyword list, and for each keyword, combine it with the timestamp information, which includes the start and end times, as well as other relevant metadata (such as volume, speech rate, etc., but here we mainly focus on the start and end times). Add the timestamped keywords to the keyword sequence list. Each timestamped keyword can be represented as a dictionary or tuple.

[0027] S105. Based on the word segmentation results, sentiment trends, audio conversion and timestamp information obtained in steps S101 to S104, and the keyword time series, generate a digital human.

[0028] Using the MetaHuman3D model, a 3D model is initialized, and a text-to-speech (TTS) system is integrated to enable the digital human to generate speech based on text content. TTS system parameters, such as speech model, speech rate, pitch, and volume, are configured to ensure the generated speech matches the digital human's appearance. Using the word segmentation results obtained in step S101, the text content is displayed in the digital human's interactive interface as words or phrases, with extracted keywords highlighted to emphasize key points. Based on the text's emotional tone determined in step S102, the digital human's facial expressions and movements are adjusted to express corresponding emotions. For example, if the text conveys positive emotions, the digital human can display positive gestures such as smiling and nodding; if it conveys negative emotions, it may display gestures such as frowning and shaking its head. Using the keyword time series generated in step S104, a detailed interaction plan is developed for the digital human. When the user interacts with the digital human, the digital human can mention or emphasize certain keywords at appropriate times based on the keyword time series to enhance the coherence and relevance of the interaction.

[0029] The system receives user input, such as voice, text, or gestures, parses the input, identifies the user's intent and needs, and designs the digital human's interaction logic based on the user's input and the current scenario. For example, when a user asks for information related to a certain keyword, the digital human can quickly locate the keyword in the text and provide a corresponding explanation or description. The digital human provides corresponding feedback and responses based on the interaction logic. The feedback can take the form of voice, text, facial expressions, or actions to provide a rich interactive experience.

[0030] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages: By combining dictionary and statistical methods in word segmentation, key words in the text can be extracted accurately and comprehensively, including new words and technical terms, improving the efficiency and accuracy of text processing. The dictionary-based sentiment polarity annotation method can quickly determine the sentiment tendency of the text, which helps to better understand the text content and provides an emotional basis for subsequent digital human interaction. By combining word segmentation results, sentiment tendency, audio conversion and timestamp information, as well as keyword time series, digital humans with rich interactive experiences can be generated, which can better meet user needs and improve user experience.

[0031] Example 2: Based on the keywords in Example 1, this example combines the keywords into keyword blocks. These keyword blocks are matched with the action chains composed of action sequences, making the actions more coherent and fluid. Figure 2 As shown.

[0032] S201, Based on the word segmentation results of step S101, combine the keywords into keyword blocks.

[0033] Based on the generated words or vocabulary, through the collection of a large amount of data, the co-occurrence frequency of each word with other words in the text content is statistically analyzed. Words with a co-occurrence frequency that reaches a preset frequency are combined into keyword blocks, that is, words with high co-occurrence frequency are combined into keyword blocks.

[0034] In some embodiments, the sentence "The weather will be sunny tomorrow, which is suitable for going on a picnic" is segmented into words such as "tomorrow", "weather", "sunny", "suitable", "go out", and "picnic". By analyzing the text data or using existing language knowledge bases, "weather" and "sunny" often appear together to describe the weather conditions, while "go out" and "picnic" are a combination. Therefore, "sunny weather" and "going out for a picnic" are identified as two keyword blocks.

[0035] S202, Build an action sequence database and store it in key-value pairs.

[0036] Specifically, the keys are defined, representing semantics, and are representative words of one or more synonyms. "Happy" and "joyful" belong to the same key. The values ​​are defined, and the values ​​are the corresponding action sequences, which are animation descriptions composed of multiple action frames. The key-value pairs are stored using a relational database MySQL to build an action sequence database.

[0037] S203 uses an algorithm to semantically match keyword blocks with an action sequence database.

[0038] Based on an expanded version of the thesaurus, the knowledge base contains a large number of words and their synonyms and near-synonyms, which helps to capture semantic relationships between words. Keyword blocks are queried one by one in the knowledge base to find words with similar or identical semantics. The keyword blocks are then matched with semantics in the action sequence database. If the similarity exceeds a preset threshold, the keyword block is successfully matched, and the corresponding action sequence is determined. If no matching semantic is found in the database, or if multiple semantics are matched with similarity, the final action sequence is determined according to preset rules. These preset rules include: selecting the action sequence corresponding to the semantic with the highest similarity; if multiple semantics have very high similarity and their action sequences are compatible or combinable, then considering merging these action sequences; if the optimal choice cannot be determined, it may be necessary to request more information from the user or perform manual intervention.

[0039] S204 inserts the successfully matched action sequence into the action chain according to the order in which the keyword blocks appear in the text.

[0040] Specifically, based on the keyword time series generated in step S104, the order in which the keyword blocks appear in the text is determined. Following the order of the keyword blocks, the corresponding action sequences are sequentially inserted into the action chain. The action chain consists of multiple consecutive action sequences, each representing a series of continuous actions of the digital human. During insertion, it is necessary to ensure smooth transitions between action sequences to avoid action breaks or repetitions. Numerical calculation methods can be used, including Spherical Linear Interpolation (Slerp). Slerp can be used to calculate transitional actions between two actions. Through Slerp, a series of intermediate actions can be generated, allowing one action to transition to another through a series of intermediate actions. Adding a series of intermediate actions makes the transition between the two actions more natural and fluid.

[0041] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages: By inserting successfully matched action sequences into action chains, discrete action sequences are integrated into a coherent and smooth action chain, making the digital human's movements more natural and realistic. The action chain makes the continuity between multiple actions of the digital human smoother and the transition between multiple actions more natural.

[0042] Example 3: Based on the matching between keyword blocks and action sequences, a matching factor is introduced to adjust the matching degree parameters between keyword blocks and action sequences, and the action chain is adjusted and optimized so that the digital human's action performance is more in line with the semantic and emotional requirements of the text content.

[0043] Matching factors are determined by considering influencing factors, including the importance, sentiment intensity, frequency, and position of keyword blocks in the text. For each factor, a quantitative standard is set: importance is determined based on the keyword block's role or function in the text; sentiment intensity is determined based on the results of sentiment analysis algorithms; frequency is determined based on the number of times the keyword block appears in the text; and position is determined based on the keyword block's position in paragraphs or sentences within the text. Based on these quantified influencing factors, the matching factor is calculated through weighted summation and product. Matching factor = Importance weight. Importance value + emotional intensity weight The emotional intensity value adjusts the matching parameters of the corresponding action sequence based on the matching factor value. For keyword blocks with high matching factors, the weight or duration of their corresponding action sequence can be increased to make them more prominent in the action chain; for keyword blocks with low matching factors, the weight or duration of their corresponding action sequence can be reduced to decrease their importance in the action chain.

[0044] In some embodiments, given a text containing the keyword block "anger," first, the importance of anger in the text is determined to be high, and the emotional intensity to be strong. Then, a matching factor value for anger is calculated according to a preset quantification standard and formula, where the matching factor = importance weight. Importance value + emotional intensity weight The emotional intensity value is used, with both importance and emotional intensity weights set to 1. The importance value ranges from 1 to 3 (low-medium-high), and the emotional intensity value ranges from 1 to 5 (weak-medium-strong-very-extremely strong). Anger has an importance value of 3, an emotional intensity value of 5, and a matching factor of 1. 3 + 1 5=8. Based on the value of the matching factor, adjust the matching parameters of the action sequence corresponding to anger. When the matching factor is greater than or equal to 7, increase the amplitude and duration of the action sequence. Increase the amplitude and duration of the action sequence corresponding to anger by a certain proportion to better convey the emotional information of anger.

[0045] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages: By comprehensively considering and quantifying multiple influencing factors such as the importance, emotional intensity, frequency of occurrence, and position of keyword blocks in the text, we can more accurately match keyword blocks with corresponding action sequences. Dynamically adjust action sequence parameters: Based on the value of the matching factor, dynamically adjust parameters such as the weight, amplitude, and duration of the corresponding action sequence to make key nodes in the action chain more prominent and non-key nodes weaker, thereby enhancing the overall expressiveness and communication effect.

[0046] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating a digital human, characterized in that, include: S101, Input text content, preprocess the text content, segment the text content into individual words or words, and extract keywords from the text; S102, Based on the text content, perform sentiment polarity labeling to determine the sentiment tendency of the text; S103, convert the text content into audio and generate a timestamp; S104. Based on the keywords extracted in step S101, and combined with the timestamps, generate a keyword time series. S105. Based on the word segmentation results, sentiment trends, audio conversion and timestamp information obtained in steps S101 to S104, and the keyword time series, generate a digital human.

2. The method for generating a digital human as described in claim 1, characterized in that, Based on the extracted keywords, create an empty keyword sequence list to store the keywords with timestamps. Iterate through the extracted keyword list, and for each keyword, combine it with timestamp information, which includes start and end times.

3. The method for generating a digital human as described in claim 1, characterized in that, The extracted keywords are combined into keyword blocks, and the steps for forming an action chain from these keyword blocks and action sequences are as follows: S201, Based on the word segmentation results of step S101, combine the keywords into keyword blocks; Based on a number of words or words, the co-occurrence frequency of each word with other words in the text is counted, and words with a co-occurrence frequency that reaches a preset frequency are combined into keyword blocks. S202, Build an action sequence database and store it in the form of key-value pairs; S203, use an algorithm to semantically match keyword blocks with an action sequence database; S204 inserts the successfully matched action sequence into the action chain according to the order in which the keyword blocks appear in the text.

4. The method for generating a digital human as described in claim 3, characterized in that, The keyword block is semantically matched with the action sequence database. If the similarity exceeds a preset threshold, the keyword block is successfully matched with the semantics, and the action sequence corresponding to the semantics is matched. If the keyword block cannot find a matching semantic in the database, the final action sequence is determined according to preset rules.

5. The method for generating a digital human as described in claim 1, characterized in that, Based on the keyword time sequence generated in step S104, the order in which the keyword blocks appear in the text is determined. According to the order of the keyword blocks, the corresponding action sequences are inserted into the action chain in sequence. The action chain consists of multiple consecutive action sequences, and each action sequence represents a series of consecutive actions of the digital human.

6. The method for generating a digital human as described in claim 4, characterized in that, The keyword blocks are matched with the action sequence database. The degree of matching between the keyword blocks and the action sequences is adjusted by matching factors. The matching factors are determined by influencing factors, including the importance, emotional intensity, frequency of occurrence and position of the keyword blocks in the text.

7. The method for generating a digital human as described in claim 6, characterized in that, The importance is set based on the role of the keyword block in the text, the sentiment intensity is set based on the result of the sentiment analysis algorithm, the frequency of occurrence is set based on the number of times the keyword block appears in the text, and the position is set based on the position of the keyword block in the paragraph or sentence of the text.

8. The method for generating a digital human as described in claim 6, characterized in that, The matching factor = importance weight Importance value + emotional intensity weight Emotional intensity value.

9. The method for generating a digital human as described in claim 1, characterized in that, The sentiment polarity of the text is labeled using a dictionary-based approach. Each word in the text is traversed, and its corresponding polarity in the sentiment dictionary is found. Then, the sentiment score of the entire text is calculated based on the polarity and weight of the words. Finally, the sentiment tendency of the text is determined based on the score.

Images