Digital human generation system and method based on large model

Abstract

The invention discloses a digital human generation system and method based on a large model. The system comprises a data acquisition module, a data processing module and a digital human generation module, relates to the technical field of digital human generation, and solves the technical problem of poor interaction experience between a digital human generated in the prior art and a user. According to the method, the corresponding role welcome scores are calculated based on the attention degrees of a plurality of users to each role; determining a hot role sequence for generating the digital person based on the role popularity score; extracting a plurality of role features from the original feature data of each role in the hot role sequence; inputting the role features and the role knowledge base into a digital human generation model to obtain a digital human corresponding to the role features; according to the method, the corresponding knowledge points conforming to the role features are selected from the knowledge database to construct the role knowledge base, so that the knowledge points in the role knowledge base of each role are closer to role setting, and the interactive experience between a user and a digital human is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital human generation, and in particular relates to a digital human generation system and method based on a large model. Background Art

[0002] With the rapid development of artificial intelligence technology, digital companions, as an innovative smart product, are gradually entering people's lives. It integrates a number of advanced technologies such as natural language processing, emotional computing, and human-computer interaction, aiming to provide users with personalized and intelligent companionship and services. The emergence of digital companions not only enriches people's social methods, but also brings users a more convenient and efficient life experience. Therefore, exploring the patent background technology of digital companions is of great significance to promoting their further development.

[0003] The existing technology obtains the characteristic information of several characters and generates corresponding digital humans through a large model based on the characteristic information of the characters; however, a large amount of computing resources are required to generate digital humans. The digital human generation method of the existing technology is difficult to allocate the computing resources required for generating digital humans according to the popularity of the characters corresponding to the digital humans, which makes the generation of digital humans time-consuming and inefficient, resulting in a poor interaction experience between users and digital humans.

[0004] The present invention provides a digital human generation system and method based on a large model to solve the above technical problems. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a digital human generation system and method based on a large model, which is used to solve the technical problem that the digital human generation in the prior art is time-consuming and inefficient, resulting in a poor interactive experience between the user and the digital human.

[0006] To achieve the above-mentioned object, the first aspect of the present invention provides a digital human generation system based on a large model, comprising: a data processing module, and a data acquisition module and a digital human generation module connected thereto;

[0007] The data collection module is used to collect the original feature data of several roles; obtain the degree of attention of several users to each role; wherein the original feature data includes video, audio and text description; the degree of attention includes the number of comments and browsing time;

[0008] The data processing module is used to calculate the corresponding character popularity score based on the degree of attention paid by a number of users to each character; determine and generate a popular character sequence of digital humans based on the character popularity score; extract a number of character features from the original feature data of each character in the popular character sequence; select corresponding knowledge points from the knowledge database based on the character features to construct a character knowledge base; wherein the character features include image features and language features;

[0009] The digital human generation module is used to input the character characteristics and the character knowledge base into the digital human generation model to obtain a digital human corresponding to the character characteristics; wherein the digital human generation model is constructed based on the large model.

[0010] The present invention obtains the user's attention to each character, calculates the corresponding character popularity score based on the attention level of each character, and adds the characters with character popularity scores greater than a set threshold into the popular character sequence for generating digital humans, so that the corresponding digital humans can be pre-generated in advance according to the popularity of the characters, which is beneficial to improving the generation efficiency of digital humans; a number of character features are extracted from the original feature data of each character in the popular character sequence, and a character knowledge base is constructed based on the character features by selecting corresponding knowledge points from a knowledge database, and the character features and the character knowledge base are input into a digital human generation model to obtain a digital human corresponding to the character features, so that the generated digital human has knowledge that conforms to the set role, which is beneficial to improving the interactive experience between the user and the digital human.

[0011] Preferably, the step of calculating the corresponding character popularity score based on the degree of attention paid by a number of users to each character includes:

[0012] Extract the number of comments and browsing time of several users' attention to each character; filter the number of comments to obtain the real number of comments; calculate the corresponding character popularity score SHFi by the formula SHFi=a×ZPSi+b×ln(1+LSCi); where ZPSi is the number of comments on character i, LSCi is the browsing time of character i; a and b are proportional coefficients greater than 0, ln() is a logarithmic function with natural constants as the base; i=1, 2,…, n, n is the total number of characters.

[0013] Preferably, the screening of the number of comments includes:

[0014] Extract the comments of several users on each character; determine whether the number of comments with the same content on the same character is less than twice; if yes, mark the true label of the corresponding comment as 1; if no, mark the true label of the corresponding comment as 0; mark the sum of all the comments with the true label of 1 as the true number of comments.

[0015] The present invention determines whether users have comments with the same content on each character, removes the number of comments with the same content to obtain the real number of comments, so that the subsequent real number of comments can more accurately reflect the real popularity of the character when calculating the popularity score of the character, which is beneficial to improving the accuracy of the system in scheduling computing resources in the process of generating digital humans.

[0016] Preferably, the process of determining and generating a popular character sequence of digital humans based on character popularity scores includes:

[0017] Extract the popularity score of each character; determine whether the character popularity score is greater than a preset popularity score threshold; if yes, add the corresponding character to the popular character sequence; if no, identify the corresponding character as a common character.

[0018] Preferably, the extraction of several character features from the original feature data of each character in the popular character sequence includes:

[0019] Extract the original feature data corresponding to each character in the popular character sequence; extract the image features and language features of the corresponding character from the video, audio and text descriptions in the original feature data; integrate the image features and language features into character features; among which, the image features include age, appearance, gender, personality, occupation and clothing.

[0020] Preferably, the selecting corresponding knowledge points from the knowledge database based on the role characteristics to construct the role knowledge base includes:

[0021] The role features corresponding to several roles are extracted, and the corresponding knowledge points that meet the role features are selected from the knowledge database to construct a role knowledge base; wherein the knowledge database includes the knowledge points required by all roles.

[0022] The present invention constructs a role knowledge base by selecting corresponding knowledge points that meet the role characteristics from a knowledge database, so that the knowledge points in the role knowledge base of each role are more in line with the role setting of the digital human generated subsequently, which is conducive to improving the interactive experience between the user and the digital human.

[0023] Preferably, the digital human generation model is constructed based on a large model, including:

[0024] Extracting a number of role features and a role knowledge base corresponding to the role features; integrating the number of role features and the role knowledge base corresponding to the role features into a number of sets of training data; training a large model using the number of sets of training data, and marking the trained large model as a digital human generation model; wherein the large model includes Diffusion Avatars and PoseGPT.

[0025] A second aspect of the present invention provides a method for generating a digital human based on a large model, comprising:

[0026] S1: Collect the original characteristic data of several roles; obtain the attention level of several users to each role;

[0027] S2: Calculate the popularity score of each character based on the attention paid by a number of users to each character;

[0028] S3: Determine the popular role sequence for generating digital humans based on role popularity scores;

[0029] S4: extracting several role features from the original feature data of each role in the popular role sequence;

[0030] S5: Select corresponding knowledge points from the knowledge database based on role characteristics to build a role knowledge base;

[0031] S6: Input the role characteristics and the role knowledge base into the digital human generation model to obtain a digital human corresponding to the role characteristics.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The present invention obtains the user's attention to each character, calculates the corresponding character popularity score based on the attention level of each character, and adds the characters with the character popularity score greater than a set threshold into the popular character sequence for generating digital humans, so that computing resources can be allocated according to the popularity of the characters for pre-generating corresponding digital humans, which is beneficial to improving the generation efficiency of digital humans; a number of character features are extracted from the original feature data of each character in the popular character sequence, and a character knowledge base is constructed based on the character features by selecting corresponding knowledge points from a knowledge database, and the character features and the character knowledge base are input into a digital human generation model to obtain a digital human corresponding to the character features, so that the generated digital human has knowledge that conforms to the set role, which is beneficial to improving the interactive experience between the user and the digital human. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 It is an overall flow chart of the method for generating digital human based on a large model of the present invention;

[0036] Figure 2It is a schematic diagram of the principle of the digital human generation system based on a large model of the present invention. DETAILED DESCRIPTION

[0037] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] See also Figure 1-Figure 2 , the first aspect of the present invention provides a digital human generation system based on a large model, comprising: a data processing module, and a data acquisition module and a digital human generation module connected thereto;

[0039] Data collection module: used to collect the original feature data of several roles; obtain the degree of attention of several users to each role; the original feature data includes video, audio and text description; the degree of attention includes the number of comments and browsing time;

[0040] Data processing module: used to calculate the corresponding character popularity score based on the degree of attention paid by a number of users to each character; determine and generate a popular character sequence of digital humans based on the character popularity score; extract a number of character features from the original feature data of each character in the popular character sequence; select corresponding knowledge points from the knowledge database based on the character features to construct a character knowledge base; wherein the character features include image features and language features;

[0041] Digital human generation module: used to input role characteristics and role knowledge base into the digital human generation model to obtain a digital human corresponding to the role characteristics; wherein the digital human generation model is constructed based on the large model.

[0042] The present invention obtains the user's attention to each character and calculates the corresponding character popularity score based on the attention level of each character. The characters whose character popularity score is greater than a set threshold are added to the popular character sequence for generating digital humans, so that the corresponding digital humans can be pre-generated in advance according to the popularity of the characters. This not only improves the efficiency of digital human generation, but also ensures the reasonable allocation of resources and avoids unnecessary calculation and storage costs.

[0043] Furthermore, the present invention extracts several character features from the original feature data of each character in the popular character sequence, and selects corresponding knowledge points from the knowledge database based on these features to construct a character knowledge base. The character features and the character knowledge base are input into the digital human generation model to generate a digital human that matches the character features. The generated digital human not only has the knowledge that matches the set role, but also can significantly improve the interactive experience between the user and the digital human, thereby enhancing user satisfaction and the practicality of the system.

[0044] In this embodiment, the corresponding character popularity score is calculated based on the degree of attention paid by a number of users to each character, including:

[0045] Extract the number of comments and browsing time of several users' attention to each character; filter the number of comments to obtain the real number of comments; calculate the corresponding character popularity score SHFi by the formula SHFi=a×ZPSi+b×ln(1+LSCi); where ZPSi is the real number of comments on character i, and LSCi is the browsing time of character i; a and b are proportional coefficients greater than 0, and the values ​​of a and b are set by experts in related industries based on experience, and ln() is a logarithmic function with a natural constant as the base; i=1, 2,…, n, and n is the total number of characters.

[0046] Exemplarily, the proportional coefficients a=0.1 and b=3 are set; the number of comments of character 1 ZPS1=130 times, and the browsing time of character 1 LSC1=80min; the character popularity score SHF1≈26.18 of character 1 is calculated by the formula.

[0047] It is worth noting that the higher the number of comments in the user's attention to each character and the longer the browsing time, the higher the calculated character popularity score; in addition, since some users forget to close the character interface and browse the same character interface for a long time, the present invention arranges that when the browsing time is longer, the growth rate of the corresponding character popularity score slows down, thereby reducing the impact of invalid browsing time on the actual character popularity score.

[0048] The present invention obtains the degree of attention of several users to each character, and calculates the character popularity score through a dimensionless formula based on the number of comments and browsing time in the degree of attention, so that the calculated character popularity score can accurately represent the popularity of the relevant character, which is convenient for the subsequent determination of the popular character sequence according to the character popularity score, thereby facilitating the subsequent system to allocate computing resources according to the popular character sequence for pre-generating corresponding digital humans, thereby facilitating the generation efficiency of digital humans.

[0049] In this embodiment, the number of comments is screened, including:

[0050] Extract the comments of several users on each character; determine whether the number of comments with the same content on the same character is less than twice; if yes, mark the true label of the corresponding comment as 1; if no, mark the true label of the corresponding comment as 0; mark the sum of all the comments with the true label of 1 as the true number of comments.

[0051] The present invention determines whether users have comments with the same content on each character, removes the number of comments with the same content to obtain the real number of comments, so that the subsequent real number of comments can more accurately reflect the real popularity of the character when calculating the popularity score of the character, which is beneficial to improving the accuracy of the system in scheduling computing resources in the process of generating digital humans.

[0052] In this embodiment, determining and generating a popular character sequence of digital humans based on character popularity scores includes:

[0053] Extract the popularity score of each character; determine whether the character popularity score is greater than a preset popularity score threshold; if yes, add the corresponding character to the popular character sequence; if no, identify the corresponding character as a common character.

[0054] Exemplarily, the role popularity score SHF1 of role 1 is set to 26.18, and the preset popularity score threshold is 20; because the role popularity score SHF1 of role 1 is greater than the preset popularity score threshold, role 1 is added to the popular role sequence.

[0055] In this embodiment, several character features are extracted from the original feature data of each character in the popular character sequence, including:

[0056] Extract the original feature data corresponding to each character in the popular character sequence; extract the image features and language features of the corresponding character from the video, audio and text descriptions in the original feature data; integrate the image features and language features into character features; among which, the image features include age, appearance, gender, personality, occupation and clothing.

[0057] In this embodiment, corresponding knowledge points are selected from the knowledge database based on the role characteristics to construct a role knowledge base, including:

[0058] The role features corresponding to several roles are extracted, and the corresponding knowledge points that meet the role features are selected from the knowledge database to construct a role knowledge base; wherein the knowledge database includes the knowledge points required by all roles.

[0059] In this embodiment, the digital human generation model is constructed based on the large model, including:

[0060] Extract a number of role features and a role knowledge base corresponding to the role features; integrate the number of role features and the role knowledge base corresponding to the role features into a number of groups of training data; divide 80% of the number of groups of training data into standard training data and 20% into standard verification data; use the standard training data to train a large model, use standard verification data to verify the trained large model, adjust the parameters of the large model according to the verification results, and mark the trained large model as a digital human generation model; wherein the large model includes Diffusion Avatars and PoseGPT.

[0061] A second aspect of the present invention provides a method for generating a digital human based on a large model, comprising:

[0062] S1: Collect the original characteristic data of several roles; obtain the attention level of several users to each role;

[0063] S2: Calculate the popularity score of each character based on the attention paid by a number of users to each character;

[0064] S3: Determine the popular role sequence for generating digital humans based on role popularity scores;

[0065] S4: extracting several role features from the original feature data of each role in the popular role sequence;

[0066] S5: Select corresponding knowledge points from the knowledge database based on role characteristics to build a role knowledge base;

[0067] S6: Input the role characteristics and the role knowledge base into the digital human generation model to obtain a digital human corresponding to the role characteristics.

[0068] Part of the data in the above formula is calculated by removing the dimension and taking its numerical value. The formula is a formula closest to the actual situation obtained by software simulation of a large amount of collected data; the preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained through simulation of a large amount of data.

[0069] Working principle of the present invention:

[0070] The present invention collects original feature data of several role personnel; obtains the degree of attention of several users to each role; calculates the corresponding role popularity score based on the degree of attention of several users to each role; determines a popular role sequence for generating digital humans based on the role popularity score; extracts several role features from the original feature data of each role in the popular role sequence; selects corresponding knowledge points from a knowledge database based on the role features to construct a role knowledge base; inputs the role features and the role knowledge base into a digital human generation model to obtain a digital human corresponding to the role features.

[0071] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A digital human generation system based on a large model, comprising: The data processing module, and the data acquisition module and digital human generation module connected thereto are characterized in that: The data collection module is used to collect the original feature data of several roles; obtain the degree of attention of several users to each role; wherein the original feature data includes video, audio and text description; the degree of attention includes the number of comments and browsing time; The data processing module is used to calculate the corresponding character popularity score based on the degree of attention paid by a number of users to each character; determine and generate a popular character sequence of digital humans based on the character popularity score; extract a number of character features from the original feature data of each character in the popular character sequence; select corresponding knowledge points from the knowledge database based on the character features to construct a character knowledge base; wherein the character features include image features and language features; The digital human generation module is used to input the character characteristics and the character knowledge base into the digital human generation model to obtain the corresponding digital human; wherein the digital human generation model is constructed based on the large model.

2. A digital human generation system based on a large model according to claim 1, characterized in that: The calculation of the corresponding character popularity score based on the degree of attention paid by a number of users to each character includes: Extract the number of comments and browsing time of several users' attention to each character; filter the number of comments to obtain the real number of comments; calculate the corresponding character popularity score SHFi by the formula SHFi=a×ZPSi+b×ln(1+LSCi); where ZPSi is the real number of comments for character i, LSCi is the browsing time of character i; a and b are proportional coefficients greater than 0, ln() is a logarithmic function with a natural constant as the base; i=1, 2,…, n, n is the total number of characters.

3. A digital human generation system based on a large model according to claim 2, characterized in that: The screening of the number of comments includes: Extract the comments of several users on each character; determine whether the number of comments with the same content on the same character is less than twice; if yes, mark the true label of the corresponding comment as 1; if no, mark the true label of the corresponding comment as 0; mark the sum of all the comments with the true label of 1 as the true number of comments.

4. A digital human generation system based on a large model according to claim 1, characterized in that: The method of determining and generating a popular character sequence of digital humans based on character popularity scores includes: Extract the popularity score of each character; determine whether the character popularity score is greater than a preset popularity score threshold; if yes, add the corresponding character to the popular character sequence; if no, identify the corresponding character as a common character.

5. The digital human generation system based on a large model according to claim 1, characterized in that: The method extracts several character features from the original feature data of each character in the popular character sequence, including: Extract the original feature data corresponding to each character in the popular character sequence; extract the image features and language features of the corresponding character from the video, audio and text descriptions in the original feature data; integrate the image features and language features into character features; among which, the image features include age, appearance, gender, personality, occupation and clothing.

6. The digital human generation system based on a large model according to claim 1, characterized in that: The step of selecting corresponding knowledge points from a knowledge database based on role characteristics to build a role knowledge base includes: The role features corresponding to several roles are extracted, and the corresponding knowledge points that meet the role features are selected from the knowledge database to construct a role knowledge base; wherein the knowledge database includes the knowledge points required by all roles.

7. The digital human generation system based on a large model according to claim 1, characterized in that: The digital human generation model is constructed based on a large model, including: Extracting a number of role features and a role knowledge base corresponding to the role features; integrating the number of role features and the role knowledge base corresponding to the role features into a number of sets of training data; training a large model using the number of sets of training data, and marking the trained large model as a digital human generation model; wherein the large model includes Diffusion Avatars and PoseGPT.

8. A method for generating a digital human based on a large model, based on the operation of a digital human generation system based on a large model as claimed in any one of claims 1 to 7, characterized in that: include: S1: Collect the original feature data of several role personnel; Get the degree of attention of several users to each role; S2: Calculate the popularity score of each character based on the attention paid by a number of users to each character; S3: Determine the popular role sequence for generating digital humans based on role popularity scores; S4: extracting several role features from the original feature data of each role in the popular role sequence; S5: Select corresponding knowledge points from the knowledge database based on role characteristics to build a role knowledge base; S6: Input the role characteristics and role knowledge base into the digital human generation model to obtain the corresponding digital human.

Images