Digital human control method, electronic equipment and program product

Abstract

The invention discloses a digital person control method, electronic equipment and a program product. The method comprises the following steps: determining the gas consumption of a digital person for broadcasting a first voice segment; determining the residual gas volume of the digital human; determining the total inspiration amount of the digital person according to the gas consumption, the residual gas amount and the digital person gas amount; wherein the digital popularity is related to the estimated vital capacity of the digital person; and according to the relationship between the total inspiration amount and the digital popularity amount, controlling the breathing and / or ventilation action when the digital person broadcasts the first voice segment. According to the embodiment of the invention, the digital person can have breathing and / or ventilation actions during voice broadcasting, so that the credibility and attraction of the digital person are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a digital human control method, electronic equipment, and program product. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, digital humans (such as virtual anchors) are increasingly being used in live broadcasts and news reporting. Through computer-generated images and synthesized voices, digital humans provide viewers with a rich audio-visual experience and are widely used in news broadcasts, online education, and live entertainment broadcasts.

[0003] However, in related technologies, digital humans do not breathe or exchange air when making voice broadcasts. This unnatural performance not only affects the viewing experience, but is also easily identified as false or mechanical performance, affecting the credibility and attractiveness of the digital humans. Summary of the Invention

[0004] The embodiments of the present invention provide a digital human control method, electronic equipment, and program product to solve the problem that the existing digital human does not breathe or exchange air when performing voice broadcasts, which affects the credibility and attractiveness of the digital human.

[0005] In a first aspect, an embodiment of the present invention provides a digital human control method, comprising:

[0006] Determine the gas consumption of the digital human in broadcasting the first voice segment;

[0007] Determining the remaining amount of air in the digital broadcast;

[0008] Determine the total amount of inhalation of the digital person according to the gas consumption, the remaining gas volume and the digital person's gas volume; wherein the digital person's gas volume is related to the estimated vital capacity of the digital person;

[0009] According to the relationship between the total amount of inhaled air and the digital human's breath volume, the breathing and / or ventilation movements of the digital human when reading the first voice segment are controlled.

[0010] Optionally, determining the gas consumption of the digital human broadcasting the first voice segment includes:

[0011] Dividing the first speech segment into a plurality of speech sub-segments;

[0012] The air consumption is determined according to the volume and exhalation coefficient of the speech sub-segment.

[0013] Optionally, determining the total amount of air inhaled by the digital person according to the gas consumption, the remaining gas volume and the gas volume of the digital person includes:

[0014] Determining a first gas volume value according to the digital popularity volume, the first weight coefficient, and the remaining gas volume;

[0015] determining a second gas volume value according to the gas consumption and the remaining gas volume;

[0016] determining a maximum gas volume value between the first gas volume value and the second gas volume value;

[0017] The total amount of inhalation is determined according to the maximum air volume value, the digital air volume and the remaining air volume.

[0018] Optionally, determining the total amount of inhalation according to the maximum air volume value, the digital air volume, and the remaining air volume includes:

[0019] Determine a third popularity value according to the digital popularity and the second weight coefficient;

[0020] Determining a minimum gas volume value between the maximum gas volume value and the third gas volume value;

[0021] Determining a fourth popularity value according to the digital popularity and a third weight coefficient, wherein the sum of the second weight coefficient and the third weight coefficient is 1;

[0022] The total inhaled air volume is determined according to the minimum air volume value, the fourth air volume value, and the remaining air volume.

[0023] Optionally, controlling the breathing and / or ventilation action of the digital human when broadcasting the first voice segment according to the relationship between the total amount of inhalation and the digital human's breath volume includes:

[0024] Determining a breathing control parameter according to the relationship between the total amount of inhaled air and the digital breath volume, the breathing control parameter comprising at least one of the following: a nostril expansion degree control parameter, a mouth expansion degree control parameter, a chest and abdomen expansion degree control parameter, and an inhalation volume control parameter;

[0025] The breathing control parameters are input into the breathing model of the digital human, so that the breathing model controls the breathing and / or ventilation movements of the digital human according to the breathing control parameters.

[0026] Optionally, determining the breathing control parameter according to the relationship between the total amount of inspiration and the digital breath volume includes:

[0027] In the case where the total amount of inhaled air is less than a preset air volume value, determining the breathing control parameters includes: the nostril expansion degree control parameter and the chest and abdomen expansion degree control parameter; or

[0028] When the total amount of inhaled air is greater than or equal to the preset air volume value, determining the breathing control parameters includes: the nostril expansion degree control parameter, the mouth expansion degree control parameter, the chest and abdomen expansion degree control parameter and the inhalation volume control parameter.

[0029] Optionally, determining the breathing control parameter according to the relationship between the total amount of inspiration and the digital breath volume includes:

[0030] Determine the inhalation pause time T according to the relationship between the total inhalation amount and the digital breath volume;

[0031] determining the cumulative inspiratory volume at the first moment according to the inspiratory time t at the first moment and the pause time T;

[0032] A breathing control parameter at the first moment is determined according to the accumulated inspiratory volume at the first moment.

[0033] Optionally, determining the inhalation pause time T according to the relationship between the total inhalation amount and the digital breath volume includes:

[0034] When the total amount of inhaled air is less than a preset air volume value, the pause time T is determined to be a fixed sentence pause time; or

[0035] When the total amount of inhalation is greater than or equal to the preset air volume value, the pause time T is determined according to the fixed sentence pause time, the digital air volume and the total amount of inhalation.

[0036] Optionally, determining the pause time T according to the fixed sentence pause time, the digital popularity and the total amount of inhalation includes:

[0037] Determining a ratio of the fixed statement pause time to the preset gas volume value;

[0038] The pause time T is determined according to the product of the ratio and the total amount of inhaled air.

[0039] Optionally, determining the cumulative inhaled volume at the first moment based on the inhaled time t and the pause time T at the first moment includes:

[0040] When the ratio of the inhaled time t to the pause time T is less than a first threshold, the cumulative inhaled volume is determined according to a first functional relationship, wherein the instantaneous inhalation rate of the digital human is defined as a first rate in the first functional relationship;

[0041] When the ratio of the inhaled time t to the pause time T is greater than or equal to a first threshold, the accumulated inhaled volume is determined according to a second functional relationship, wherein the instantaneous inhalation rate of the digital human is defined as a second rate in the second functional relationship;

[0042] The first rate and the second rate are different.

[0043] Optionally, determining the breathing control parameter according to the accumulated inspiratory volume at the first moment includes:

[0044] determining the chest and abdomen expansion degree control parameter according to the accumulated inspiratory volume; and / or,

[0045] Determine the instantaneous inhalation speed at the first moment based on the accumulated inhalation volume at the first moment; and determine at least one of the nostril expansion degree control parameter, the mouth expansion degree control parameter, and the inhalation volume control parameter based on the instantaneous inhalation speed.

[0046] In a second aspect, an embodiment of the present invention further provides a digital human control device, comprising:

[0047] A first determining module is used to determine the gas consumption of the digital human broadcasting the first voice segment;

[0048] A second determining module is used to determine the remaining gas volume of the digital human;

[0049] A third determining module is configured to determine the total amount of inhalation of the digital person based on the gas consumption, the remaining gas volume, and the digital person's gas volume; wherein the digital person's gas volume is related to the digital person's estimated vital capacity;

[0050] The processing module is used to control the breathing and / or ventilation action of the digital human when it broadcasts the first voice segment according to the relationship between the total amount of inhalation and the amount of the digital human's breath.

[0051] In a third aspect, an embodiment of the present invention further provides an electronic device comprising: a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor; the processor is configured to read the program in the memory to implement the steps of the digital human control method as described above.

[0052] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the digital human control method as described above are implemented.

[0053] In a fifth aspect, an embodiment of the present invention further provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps in the digital human control method as described above.

[0054] In an embodiment of the present invention, the amount of air used by the digital human when broadcasting the first voice segment is determined; the remaining air volume of the digital human is determined; the total amount of air inhaled by the digital human is determined based on the air used, the remaining air volume, and the digital human air volume; wherein the digital human air volume is related to the estimated vital capacity of the digital human; and based on the relationship between the total amount of air inhaled and the digital human air volume, the breathing and / or ventilation movements of the digital human when broadcasting the first voice segment are controlled, so that the digital human can perform breathing and / or ventilation movements when broadcasting voice, thereby improving the credibility and attractiveness of the digital human. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in describing the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0056] Figure 1 is a flow chart of a digital human control method provided by an embodiment of the present invention;

[0057] Figure 2 This is one of the breathing and ventilation diagrams provided by the embodiment of the present invention;

[0058] Figure 3 This is the second schematic diagram of breathing and ventilation provided by an embodiment of the present invention;

[0059] Figure 4 is a structural diagram of a digital human control device provided by an embodiment of the present invention;

[0060] Figure 5 It is a schematic diagram of the hardware structure of the electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0062] See also Figure 1 , Figure 1is a flow chart of a digital human control method provided by an embodiment of the present invention. Figure 1 As shown, the following steps are included:

[0063] Step 101: Determine the gas consumption of the digital human broadcasting the first voice segment.

[0064] Among them, digital human is a digital character image, which is rendered based on a three-dimensional model, including facial expressions, body movements, and clothing dynamics.

[0065] In specific implementation, after obtaining the text to be broadcast, the text to be broadcast is first pre-processed by word segmentation, part-of-speech tagging and segmentation tagging; then the sentences in the text are subjected to grammatical analysis and semantic analysis through natural language processing (NLP) to identify the emotional information and pause information in the sentences; based on the emotional information and pause information, the text to be broadcast is segmented to generate multiple natural and fluent voice segments with various emotions; among which the first voice segment is any one of the multiple voice segments.

[0066] Optionally, the volume of the voice segment can be used to determine the amount of air used by the digital human to read the voice segment. It should be noted that the volume of the audio will change according to the emotion of the reading / reciting. Figure 2 In the sentence "Spring tide is your style", the emotion is more excited, some notes will be more passionate, and the volume will be different.

[0067] Step 102: Determine the remaining gas volume of the digital human.

[0068] Among multiple consecutive voice segments, if the first voice segment is not the very first one, the remaining air volume of the digital human is the remaining air volume after the second voice segment is broadcast. The second voice segment is the voice segment immediately preceding the first voice segment. If the first voice segment is the very first one, the remaining air volume can also be understood as the remaining available air volume before the first voice segment is broadcast.

[0069] Optionally, the remaining air volume after the digital human finishes broadcasting the second voice segment can be equal to the total air volume before the digital human broadcasts the second voice segment minus the air volume used in the second voice segment; the air volume used in the second voice segment is determined according to the volume of the second voice segment.

[0070] Step 103: Determine the total amount of inhalation of the digital person according to the gas consumption, the remaining gas volume and the digital person's gas volume; wherein the digital person's gas volume is related to the estimated vital capacity of the digital person.

[0071] In this step, the total amount of air inhaled by the digital human can be understood as the amount of air that the digital human needs to inhale in addition to the existing remaining air volume before the first voice segment is broadcast.

[0072] The digital person's breath volume is positively correlated with the digital person's estimated vital capacity. For example, the digital person's breath volume Q = F × λ, where λ is the breath volume coefficient and F is the digital person's estimated vital capacity. Optionally, the digital person's estimated vital capacity can be estimated using a pre-trained vital capacity assessment model based on the digital person's gender, exercise habits, broadcasting training, occupation, and other characteristics.

[0073] Step 104: According to the relationship between the total amount of inhaled air and the digital human's breath volume, the breathing and / or ventilation movements of the digital human when reading the first voice segment are controlled.

[0074] The breathing and / or ventilation movements include but are not limited to at least one of the following: nostril expansion movement, mouth expansion movement, chest expansion movement, inhalation sound, and exhalation sound.

[0075] It should be noted that common breathing techniques include:

[0076] Normal Qi replenishment;

[0077] the deep breath just before the onset of climax and intense emotion;

[0078] Holding your breath before revealing the suspense or a more intense outburst.

[0079] For example, Figure 2 In the broadcast of the sentence "You come from the snow-capped mountains, the spring tide is your style; you rush to the East China Sea...", the phrase "Spring tide is your style" is emotionally charged and loud, requiring a large amount of air. At this point, an algorithm is needed to simulate a large inhalation process, utilizing expansion of the mouth and nose, chest and abdomen, and inhalation sounds. After the broadcast of "Spring tide is your style," the tone of "You rush to the East China Sea" is relatively calm, requiring only the algorithm to simulate expansion of the nostrils, chest and abdomen.

[0080] For example, breath holding is often used before a stronger outburst of emotion, or to draw the audience's attention to the following words. After NLP recognizes the emotional information, the timing of breath holding can be determined based on the emotional information. During the breath holding period, the breathing state is maintained to simulate a short period of apnea, thereby attracting attention or thinking. Figure 3 In the video, hold your breath briefly after reading the voice segment "What happened next?", and then apply the same breathing algorithm after exhaling before starting to read the next voice segment. After exhaling, you can reasonably control your breathing and ventilation techniques according to the amount of air required later.

[0081] In the above embodiment, the digital person can perform breathing and / or ventilation movements when performing voice broadcasting, thereby improving the credibility and attractiveness of the digital person.

[0082] In some embodiments of the present application, in step 101, determining the gas consumption of the digital human broadcasting the first voice segment includes:

[0083] Dividing the first speech segment into a plurality of speech sub-segments;

[0084] The air consumption is determined according to the volume and exhalation coefficient of the speech sub-segment.

[0085] For example, assuming that the audio length of the first speech segment is L (in seconds), the first speech segment is evenly segmented according to the time θ milliseconds to obtain n speech sub-segments, that is, n=L×1000 / θ.

[0086] For example, speech segments can be segmented by detecting silent parts of the audio signal, volume changes, or specific speech features, such as syllables and word boundaries.

[0087] For example, according to the volume V of the speech sub-segment i and the exhalation coefficient γ, to determine the air consumption Q of the digital human broadcasting the first voice segment next .Right now, i is a positive integer between 1 and n.

[0088] The exhalation coefficient can be estimated by analyzing speech characteristics (such as pitch, duration, and phonation method). The exhalation coefficient may be a predefined value or predicted based on audio characteristics using a machine learning model.

[0089] In this embodiment, the process of dividing the first speech segment into multiple speech sub-segments and determining the air consumption based on the volume and expiratory coefficient allows for more precise analysis of the volume changes and vocal characteristics of each segment through detailed audio analysis and dynamic parameter adjustment, thereby improving the accuracy of air consumption calculation. For example, the expiratory coefficient may be higher in excited or fast speech, while lower in calm or slow speech.

[0090] In some embodiments of the present application, in step 103, determining the total amount of inhalation of the digital person according to the gas consumption, the remaining gas volume, and the gas volume of the digital person includes:

[0091] Determining a first gas volume value according to the digital popularity volume, the first weight coefficient, and the remaining gas volume;

[0092] determining a second gas volume value according to the gas consumption and the remaining gas volume;

[0093] determining a maximum gas volume value between the first gas volume value and the second gas volume value;

[0094] The total amount of inhalation is determined according to the maximum air volume value, the digital air volume and the remaining air volume.

[0095] For example, the first gas volume value = a1×QQ r , a1 is the first weight coefficient, Q is the digital popularity, Q r Is the remaining gas volume; the second gas volume value = Q next -Q r , Q next Report the gas consumption of the first voice segment to the digital human, Q r is the remaining gas volume; the maximum gas volume value = max(a1×QQ r , Q next -Q r ).

[0096] In the above embodiment, by combining the digital person's gas volume, gas consumption and remaining gas volume, multiple factors can be comprehensively considered to ensure the accuracy and rationality of the calculation. In addition, the total amount of inhaled gas can be dynamically adjusted according to the real-time gas consumption and remaining gas volume to adapt to different dialogue scenarios or situational changes, thereby making the digital person's performance more natural.

[0097] In some embodiments of the present application, determining the total amount of inhalation according to the maximum air volume value, the digital air volume, and the remaining air volume includes:

[0098] Determine a third popularity value according to the digital popularity and the second weight coefficient;

[0099] Determining a minimum gas volume value between the maximum gas volume value and the third gas volume value;

[0100] Determining a fourth popularity value according to the digital popularity and a third weight coefficient, wherein the sum of the second weight coefficient and the third weight coefficient is 1;

[0101] The total inhaled air volume is determined according to the minimum air volume value, the fourth air volume value, and the remaining air volume.

[0102] For example, the total amount of inhaled air can be determined according to Formula 1; Formula 1 is: Q ine =min(max(first gas volume value, second gas volume value), a2×Q)+(1-a2)×QQ r ; Wherein, the third gas volume value = a2×Q, Q ine is the total amount of air inhaled by the digital human before it broadcasts the first voice segment, Q is the air volume of the digital human, Q r is the remaining gas volume, a2 is the second weight coefficient, and the third weight coefficient = 1-a2.

[0103] For example, a1=0.5, a2=0.7. That is, Q ine =min(max(0.5×QQ r , Q next -Q r ), 0.7×Q)+0 / 3×QQ r Here, the values ​​of a1 and a2 are just examples and can be adjusted according to actual needs.

[0104] It should be noted that Formula 1 can also be modified based on actual needs, and there is no limitation here.

[0105] It should be noted that “Q next -Q r " represents the difference between the air volume required for the first voice segment and the remaining air volume, ensuring that the digital human can fully broadcast the current segment. It can dynamically adapt to different voice lengths to avoid voice interruptions due to insufficient air volume, ensuring that the digital human can fully broadcast the current segment. "a1×Q" represents the theoretical inhalation demand calculated based on the digital human's standard vital capacity, further subtracting the remaining air volume to ensure that there is no over-inhalation, prevent the digital human from shortness of breath or lack of air, and ensure basic breathing stability. The formula max(a1×QQ r , Q next -Q r ) By comparing the basic air volume of the digital human and the air volume required for voice and taking the larger value, it can be ensured that the digital human will not affect the smoothness of the broadcast due to insufficient air volume.

[0106] By formula Q ine =min(max(a1×QQ r , Q next -Q r ), a2×Q) can limit the maximum inhaled volume to no more than "a2×Q". In other words, even if a larger volume is needed, it cannot exceed a2×Q, avoiding excessive inhalation and causing unnatural breathing rhythms. In this way, the physiological limitations of human breathing can be simulated, making the digital human breathe more naturally.

[0107] “(1-a2)×Q” represents the additional gas volume, which ensures that the digital human will not frequently breathe due to low remaining gas volume. r "Deduct the remaining gas volume to avoid repeated calculation. Therefore, through the formula "(1-a2)×QQ r "It can optimize breathing rhythm, allowing the digital human to maintain a stable air supply when reading long sentences.

[0108] In the above embodiment, by dynamically adjusting the inhalation volume, it is possible to ensure that long sentences have sufficient air volume and short sentences do not inhale too much air. At the same time, the minimum air supply demand is ensured and the maximum inhalation volume is limited, making the breathing behavior of the digital human more natural and more in line with physiological laws during broadcasting.

[0109] In some embodiments of the present application, controlling the breathing and / or ventilation action of the digital human when broadcasting the first voice segment based on the relationship between the total amount of inhalation and the digital human's breath volume includes:

[0110] Determining a breathing control parameter according to the relationship between the total amount of inhaled air and the digital breath volume, the breathing control parameter comprising at least one of the following: a nostril expansion degree control parameter, a mouth expansion degree control parameter, a chest and abdomen expansion degree control parameter, and an inhalation volume control parameter;

[0111] The breathing control parameters are input into the breathing model of the digital human, so that the breathing model controls the breathing and / or ventilation movements of the digital human according to the breathing control parameters.

[0112] In this embodiment, BlenderShape and skeletal rigging technology can be used to create a breathing model for the digital human, ensuring that the digital human's mouth, nose, chest, and abdomen utilize breathing and ventilation techniques. Breathing control parameters are used to adjust the movement amplitude of the breathing model. Breathing control parameters introduced here include, but are not limited to, nostril expansion control parameters, mouth expansion control parameters, chest and abdomen expansion control parameters, and inhalation volume control parameters.

[0113] In some embodiments of the present application, determining the breathing control parameter according to the relationship between the total amount of inspiration and the digital breath volume includes:

[0114] In the case where the total amount of inhaled air is less than a preset air volume value, determining the breathing control parameters includes: the nostril expansion degree control parameter and the chest and abdomen expansion degree control parameter; or

[0115] When the total amount of inhaled air is greater than or equal to the preset air volume value, determining the breathing control parameters includes: the nostril expansion degree control parameter, the mouth expansion degree control parameter, the chest and abdomen expansion degree control parameter and the inhalation volume control parameter.

[0116] In the above embodiment, if the total amount of inhaled air is less than the preset volume, it indicates a relatively small amount of air, and only the expansion of the nostrils, chest, and abdomen is simulated. If the total amount of inhaled air is greater than or equal to the preset volume, it indicates a large amount of air, and the expansion of the nostrils, mouth, chest, and abdomen needs to be simulated, and the inhalation sound needs to be played. This allows for dynamic adjustment of the digital human's breathing animation and sound effects, ensuring that the digital human's performance matches the environment and context.

[0117] In some embodiments of the present application, determining the breathing control parameter according to the relationship between the total amount of inspiration and the digital breath volume includes:

[0118] Determine the inhalation pause time T according to the relationship between the total inhalation amount and the digital breath volume;

[0119] determining the cumulative inspiratory volume at the first moment according to the inspiratory time t at the first moment and the pause time T;

[0120] A breathing control parameter at the first moment is determined according to the accumulated inspiratory volume at the first moment.

[0121] In the above embodiment, the pause time can be dynamically adjusted based on the relationship between the total inhaled volume and the digital person's breath volume, so that the digital person's breathing behavior is more in line with physiological laws; based on the inhaled time t and the pause time T, the cumulative inhaled volume is calculated; further, based on the cumulative inhaled volume, the breathing control parameters are calculated to adjust the digital person's breathing and ventilation behavior.

[0122] In some embodiments of the present application, determining the inhalation pause time T according to the relationship between the total inhalation volume and the digital breath volume includes:

[0123] When the total amount of inhaled air is less than a preset air volume value, the pause time T is determined to be a fixed sentence pause time; or

[0124] When the total amount of inhalation is greater than or equal to the preset air volume value, the pause time T is determined according to the fixed sentence pause time, the digital air volume and the total amount of inhalation.

[0125] The fixed sentence pause time may be understood as a normal sentence pause time, or a standard pause time required when broadcasting a specific sentence.

[0126] In the above embodiment, when the total amount of inhaled air is less than the preset air volume value, the pause time T is set to a fixed sentence pause time, so that when the air volume is insufficient, the digital human should maintain a standard pause time to maintain the stability of the digital human's breathing; when the total amount of inhaled air is greater than or equal to the preset air volume value, the pause time T will be calculated based on the fixed sentence pause time, the digital human's air volume and the total amount of inhaled air, so that when the air volume is sufficient, the pause time can be dynamically adjusted to meet the digital human's breathing needs.

[0127] In some embodiments of the present application, determining the pause time T according to the fixed sentence pause time, the digital popularity amount, and the total amount of inhalation includes:

[0128] Determining a ratio of the fixed statement pause time to the preset gas volume value;

[0129] The pause time T is determined according to the product of the ratio and the total amount of inhaled air.

[0130] For example, pause time Wherein, the preset popularity value = a4×the digital popularity value Q, a4 is the fourth weight coefficient, T 固 The fixed statement pause time.

[0131] For example, when Q ine When Q <0.4Q, a fixed time is used for inhalation, that is, the pause time T = fixed sentence pause time; when Q ine ≥0.4Q, pause time

[0132] In the above embodiment, dynamic adjustment of the pause time T is achieved by calculating the ratio of a fixed sentence pause time to a preset breath volume value and multiplying this ratio by the total amount of inhalation. This algorithm not only takes into account the breath volume state of the digital human to simulate the digital human's natural reactions under different breath volume states, but also effectively simulates physiological characteristics, thereby enhancing the naturalness of the virtual character.

[0133] In some embodiments of the present application, determining the cumulative inhaled volume at the first moment according to the inhaled time t at the first moment and the pause time T includes:

[0134] When the ratio of the inhaled time t to the pause time T is less than a first threshold, the cumulative inhaled volume is determined according to a first functional relationship, wherein the instantaneous inhalation rate of the digital human is defined as a first rate in the first functional relationship;

[0135] When the ratio of the inhaled time t to the pause time T is greater than or equal to a first threshold, the accumulated inhaled volume is determined according to a second functional relationship, wherein the instantaneous inhalation rate of the digital human is defined as a second rate in the second functional relationship;

[0136] The first rate and the second rate are different.

[0137] For example, when the ratio of the inhaled time t to the pause time T is less than the first threshold, the first functional relationship is: the cumulative inhaled volume That is, take the derivative of the first functional relationship.

[0138] For example, when the ratio of the inhaled time t to the pause time T is greater than or equal to the first threshold, the second functional relationship is the cumulative inhaled volume That is, find the derivative of the second functional relationship; where b, c, and d are calibration coefficients, Q ineis the total amount of inhalation, T is the pause time, and t is the duration of inhalation. For example, b=2, c=2, and d=2.

[0139] It can be understood that the total amount of energy of the digital person at the first moment = Q(t) + Q r .

[0140] In the above example, when the ratio of inspiratory duration t to pause time T is small, the cumulative inspiratory volume is determined using the first functional relationship, such that the cumulative inspiratory volume increases cubically with increasing inspiratory duration t. This means that in the initial stage, the response of the inspiratory volume is nonlinear, simulating a gradually accelerating inspiratory process.

[0141] When the ratio of the inhaled time t to the pause time T is large, the cumulative inhaled volume is determined by the second functional relationship, so that the cumulative inhaled volume will gradually approach the total inhaled volume Q ine ,By introducing parameters b, c and d, the nonlinear characteristics of the inhalation process can be adjusted, and ,applicable to the late stage of inhalation to better simulate the saturation state of ,inhalation.

[0142] In the above embodiment, by using different functional relationships at different stages to determine the total cumulative inhalation volume, the inhalation behavior of the digital human can be flexibly controlled at different inhalation stages (early and late). By using different rates and functional forms, the dynamic changes of the digital human during inhalation can be more accurately simulated.

[0143] In some embodiments of the present application, determining the breathing control parameter according to the accumulated inspiratory volume at the first moment includes:

[0144] determining the chest and abdomen expansion degree control parameter according to the accumulated inspiratory volume; and / or,

[0145] Determine the instantaneous inhalation speed at the first moment based on the accumulated inhalation volume at the first moment; and determine at least one of the nostril expansion degree control parameter, the mouth expansion degree control parameter, and the inhalation volume control parameter based on the instantaneous inhalation speed.

[0146] It should be pointed out that the instantaneous tensors of the nostrils, chest, abdomen, and mouth in the breathing model are all related to the inhalation speed. By calculating the slope of the function, we can get the instantaneous inhalation velocity V Q .

[0147] For example, when Q ine When Q < 0.4, only the expansion of the nostrils and chest and abdomen needs to be simulated, so that the nostril expansion degree control parameter B = V Q , chest and abdominal expansion degree control parameter K = Q(t).

[0148] For example, when Qine When ≥0.4Q, it is necessary to simulate the expansion of the nostrils, mouth, chest and abdomen, and play the inhalation sound, so that the nostril expansion degree control parameter B=V Q / 1.5, chest and abdomen expansion degree control parameter K = Q(t)*α, α is the chest and abdomen gas volume correlation coefficient; mouth expansion degree control parameter Z = V Q / 3, Inhalation volume control parameter X=V Q During the digital person's broadcast, as the words are pronounced, the amount of air decreases over time, and the chest and abdomen slowly retract.

[0149] In the above example, determining the breathing control parameter based on the accumulated inhalation volume at the first moment is only an example and is not limited thereto. The method can be modified according to actual needs.

[0150] In the above embodiment, during the digital human's broadcasting process, by controlling relevant control parameters such as the digital human's breathing sound, nostril expansion degree, chest and abdominal expansion degree, and inhalation sound, the digital human can achieve an effect closer to that of a real person, thereby increasing the digital human's sense of reality.

[0151] For example, when the digital human reads the line "You come from the snow-capped mountains, the spring tide is your elegance; you rush to the East China Sea, the surging waves are your spirit." The lines "spring tide is your elegance" and "surging waves are your spirit" are quite emotional, so the digital human takes a deep breath before "spring tide" and "surging waves" to express the excitement.

[0152] For example, when the digital human reads the sentence "Guess what happened next? ----------------------------.", the suspense is created after "What happened next?", and after "What?", the human will hold his breath to attract the audience's attention and make them think.

[0153] The embodiment of the present invention also provides a digital human control device. Figure 4 , Figure 4 This is a structural diagram of a digital human control device provided by an embodiment of the present invention. Since the principle of the problem solved by the digital human control device is similar to that of the digital human control method in the embodiment of the present invention, the implementation of the digital human control device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0154] like Figure 4 As shown, the digital human control device 400 includes:

[0155] The first determining module 401 is used to determine the gas consumption of the digital human broadcasting the first voice segment;

[0156] The second determining module 402 is used to determine the remaining gas volume of the digital human;

[0157] The third determining module 403 is configured to determine the total amount of inhalation of the digital person according to the gas consumption, the remaining gas volume, and the digital person's gas volume; wherein the digital person's gas volume is related to the estimated vital capacity of the digital person;

[0158] The processing module 404 is configured to control the breathing and / or ventilation movements of the digital human when the digital human reads the first voice segment according to the relationship between the total amount of inhalation and the amount of energy of the digital human.

[0159] Optionally, the first determining module 401 is specifically configured to:

[0160] Dividing the first speech segment into a plurality of speech sub-segments;

[0161] The air consumption is determined according to the volume and exhalation coefficient of the speech sub-segment.

[0162] Optionally, the third determining module 403 includes:

[0163] A first determining submodule, configured to determine a first gas volume value according to the digital popularity volume, a first weight coefficient, and the remaining gas volume;

[0164] a second determining submodule, configured to determine a second gas volume value according to the gas consumption and the remaining gas volume;

[0165] A third determining submodule is configured to determine a maximum gas volume value between the first gas volume value and the second gas volume value;

[0166] The fourth determining submodule is configured to determine the total amount of inhalation according to the maximum air volume value, the digital air volume, and the remaining air volume.

[0167] Optionally, the fourth determining submodule includes:

[0168] a first determining unit, configured to determine a third popularity value according to the digital popularity and a second weight coefficient;

[0169] a second determining unit, configured to determine a minimum gas volume value between the maximum gas volume value and the third gas volume value;

[0170] A third determining unit is configured to determine a fourth popularity value according to the digital popularity and a third weight coefficient, wherein the sum of the second weight coefficient and the third weight coefficient is 1;

[0171] The fourth determining unit is configured to determine the total inhaled air volume according to the minimum air volume value, the fourth air volume value, and the remaining air volume.

[0172] Optionally, the processing module 404 includes:

[0173] a first processing submodule, configured to determine a breathing control parameter based on a relationship between the total amount of inhaled air and the digital breath volume, the breathing control parameter comprising at least one of the following: a nostril expansion degree control parameter, a mouth expansion degree control parameter, a chest and abdomen expansion degree control parameter, and an inhalation volume control parameter;

[0174] The second processing submodule is used to input the breathing control parameters into the digital human breathing model, so that the breathing model controls the breathing and / or ventilation movements of the digital human according to the breathing control parameters.

[0175] Optionally, the first processing submodule includes:

[0176] The first processing unit is configured to determine, when the total amount of inhaled air is less than a preset air volume value, that the breathing control parameters include: the nostril expansion degree control parameter and the chest and abdomen expansion degree control parameter; or

[0177] The second processing unit is used to determine, when the total amount of inhaled air is greater than or equal to the preset air volume value, the breathing control parameters including: the nostril expansion degree control parameter, the mouth expansion degree control parameter, the chest and abdomen expansion degree control parameter and the inhalation volume control parameter.

[0178] Optionally, the first processing submodule includes:

[0179] a third processing unit, configured to determine an inhalation pause time T according to a relationship between the total inhalation amount and the digital breath amount;

[0180] a fourth processing unit, configured to determine a cumulative inhaled volume at the first moment based on the inhaled time t and the pause time T at the first moment;

[0181] The fifth processing unit is configured to determine a breathing control parameter at the first moment according to the accumulated inspiratory volume at the first moment.

[0182] Optionally, the third processing unit is specifically configured to:

[0183] When the total amount of inhaled air is less than a preset air volume value, the pause time T is determined to be a fixed sentence pause time; or

[0184] When the total amount of inhalation is greater than or equal to the preset air volume value, the pause time T is determined according to the fixed sentence pause time, the digital air volume and the total amount of inhalation.

[0185] Optionally, the third processing unit is further specifically configured to:

[0186] Determining a ratio of the fixed statement pause time to the preset gas volume value;

[0187] The pause time T is determined according to the product of the ratio and the total amount of inhaled air.

[0188] Optionally, the fourth processing unit is specifically configured to:

[0189] When the ratio of the inhaled time t to the pause time T is less than a first threshold, the cumulative inhaled volume is determined according to a first functional relationship, wherein the instantaneous inhalation rate of the digital human is defined as a first rate in the first functional relationship;

[0190] When the ratio of the inhaled time t to the pause time T is greater than or equal to a first threshold, the accumulated inhaled volume is determined according to a second functional relationship, wherein the instantaneous inhalation rate of the digital human is defined as a second rate in the second functional relationship;

[0191] The first rate and the second rate are different.

[0192] Optionally, the fifth processing unit is specifically configured to:

[0193] determining the chest and abdomen expansion degree control parameter according to the accumulated inspiratory volume; and / or,

[0194] Determine the instantaneous inhalation speed at the first moment based on the accumulated inhalation volume at the first moment; and determine at least one of the nostril expansion degree control parameter, the mouth expansion degree control parameter, and the inhalation volume control parameter based on the instantaneous inhalation speed.

[0195] The device provided in the embodiment of the present invention can execute the above method embodiment, and its implementation principle and technical effects are similar, so this embodiment will not be repeated here.

[0196] like Figure 5 As shown, in order to better achieve the above purpose, Figure 5 As shown, an embodiment of the present invention further provides an electronic device, including:

[0197] and a memory 520 connected to the processor 500 via a bus interface, the memory 520 being used to store programs and data used by the processor 500 when performing operations, and the processor 500 calling and executing the programs and data stored in the memory 520.

[0198] The transceiver 510 is connected to the bus interface and is used to receive and send data under the control of the processor 500. The processor 500 is used to read the program in the memory 520 to implement the following steps:

[0199] Determine the gas consumption of the digital human in broadcasting the first voice segment;

[0200] determining the remaining gas volume of the digital person;

[0201] Determine the total amount of inhalation of the digital person according to the gas consumption, the remaining gas volume and the digital person's gas volume; wherein the digital person's gas volume is related to the estimated vital capacity of the digital person;

[0202] According to the relationship between the total amount of inhaled air and the digital human's breath volume, the breathing and / or ventilation movements of the digital human when reading the first voice segment are controlled.

[0203] Among them, Figure 5 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 500 and memory represented by memory 520. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 510 may be a plurality of components, i.e., a transmitter and a transceiver, providing a unit for communicating with various other devices on a transmission medium. For different terminals, the user interface 530 may also be an interface capable of connecting external or internal devices as required, including but not limited to a keypad, display, speaker, microphone, joystick, etc. The processor 500 is responsible for managing the bus architecture and general processing, and the memory 520 may store data used by the processor 500 when performing operations.

[0204] Optionally, the processor 500 is further configured to read a program in the memory 520 to implement the following steps:

[0205] Dividing the first speech segment into a plurality of speech sub-segments;

[0206] The air consumption is determined according to the volume and exhalation coefficient of the speech sub-segment.

[0207] Optionally, the processor 500 is further configured to read a program in the memory 520 to implement the following steps:

[0208] Determining a first gas volume value according to the digital popularity volume, the first weight coefficient, and the remaining gas volume;

[0209] determining a second gas volume value according to the gas consumption and the remaining gas volume;

[0210] determining a maximum gas volume value between the first gas volume value and the second gas volume value;

[0211] The total amount of inhalation is determined according to the maximum air volume value, the digital air volume and the remaining air volume.

[0212] Optionally, the processor 500 is further configured to read a program in the memory 520 to implement the following steps:

[0213] Determine a third popularity value according to the digital popularity and the second weight coefficient;

[0214] Determining a minimum gas volume value between the maximum gas volume value and the third gas volume value;

[0215] Determining a fourth popularity value according to the digital popularity and a third weight coefficient, wherein the sum of the second weight coefficient and the third weight coefficient is 1;

[0216] The total inhaled air volume is determined according to the minimum air volume value, the fourth air volume value, and the remaining air volume.

[0217] Optionally, the processor 500 is further configured to read a program in the memory 520 to implement the following steps:

[0218] Determining a breathing control parameter according to the relationship between the total amount of inhaled air and the digital breath volume, the breathing control parameter comprising at least one of the following: a nostril expansion degree control parameter, a mouth expansion degree control parameter, a chest and abdomen expansion degree control parameter, and an inhalation volume control parameter;

[0219] The breathing control parameters are input into the breathing model of the digital human, so that the breathing model controls the breathing and / or ventilation movements of the digital human according to the breathing control parameters.

[0220] Optionally, the processor 500 is further configured to read a program in the memory 520 to implement the following steps:

[0221] In the case where the total amount of inhaled air is less than a preset air volume value, determining the breathing control parameters includes: the nostril expansion degree control parameter and the chest and abdomen expansion degree control parameter; or

[0222] When the total amount of inhaled air is greater than or equal to the preset air volume value, determining the breathing control parameters includes: the nostril expansion degree control parameter, the mouth expansion degree control parameter, the chest and abdomen expansion degree control parameter and the inhalation volume control parameter.

[0223] Optionally, the processor 500 is further configured to read a program in the memory 520 to implement the following steps:

[0224] Determine the inhalation pause time T according to the relationship between the total inhalation amount and the digital breath volume;

[0225] determining the cumulative inspiratory volume at the first moment according to the inspiratory time t at the first moment and the pause time T;

[0226] A breathing control parameter at the first moment is determined according to the accumulated inspiratory volume at the first moment.

[0227] Optionally, the processor 500 is further configured to read a program in the memory 520 to implement the following steps:

[0228] When the total amount of inhaled air is less than a preset air volume value, the pause time T is determined to be a fixed sentence pause time; or

[0229] When the total amount of inhalation is greater than or equal to the preset air volume value, the pause time T is determined according to the fixed sentence pause time, the digital air volume and the total amount of inhalation.

[0230] Optionally, the processor 500 is further configured to read a program in the memory 520 to implement the following steps:

[0231] Determining a ratio of the fixed statement pause time to the preset gas volume value;

[0232] The pause time T is determined according to the product of the ratio and the total amount of inhaled air.

[0233] Optionally, the processor 500 is further configured to read a program in the memory 520 to implement the following steps:

[0234] When the ratio of the inhaled time t to the pause time T is less than a first threshold, the cumulative inhaled volume is determined according to a first functional relationship, wherein the instantaneous inhalation rate of the digital human is defined as a first rate in the first functional relationship;

[0235] When the ratio of the inhaled time t to the pause time T is greater than or equal to a first threshold, the accumulated inhaled volume is determined according to a second functional relationship, wherein the instantaneous inhalation rate of the digital human is defined as a second rate in the second functional relationship;

[0236] The first rate and the second rate are different.

[0237] Optionally, the processor 500 is further configured to read a program in the memory 520 to implement the following steps:

[0238] determining the chest and abdomen expansion degree control parameter according to the accumulated inspiratory volume; and / or,

[0239] Determine the instantaneous inhalation speed at the first moment based on the accumulated inhalation volume at the first moment; and determine at least one of the nostril expansion degree control parameter, the mouth expansion degree control parameter, and the inhalation volume control parameter based on the instantaneous inhalation speed.

[0240] The electronic device provided in the embodiment of the present invention can execute the above method embodiment, and its implementation principle and technical effects are similar, which will not be described in detail in this embodiment.

[0241] In addition, the computer-readable storage medium of the embodiment of the present invention is used to store a computer program, and the computer program can be executed by a processor to implement the steps of the above digital human control method.

[0242] An embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, they will not be repeated here.

[0243] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection of some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0244] In addition, the functional units in various embodiments of the present invention may be integrated into a single processing unit, each unit may be physically included separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional units.

[0245] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to perform some steps of the sending and receiving methods described in various embodiments of the present invention. The aforementioned storage medium includes: a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, and other media that can store program code.

[0246] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A digital human control method, characterized in that: include: Determine the gas consumption of the digital human in broadcasting the first voice segment; determining the remaining gas volume of the digital person; Determine the total amount of inhalation of the digital person according to the gas consumption, the remaining gas volume and the digital person's gas volume; wherein the digital person's gas volume is related to the estimated vital capacity of the digital person; According to the relationship between the total amount of inhaled air and the digital human's breath volume, the breathing and / or ventilation movements of the digital human when reading the first voice segment are controlled.

2. The method according to claim 1, characterized in that The step of determining the total amount of air inhaled by the digital person according to the gas consumption, the remaining gas volume, and the gas volume of the digital person includes: Determining a first gas volume value according to the digital popularity volume, the first weight coefficient, and the remaining gas volume; determining a second gas volume value according to the gas consumption and the remaining gas volume; determining a maximum gas volume value between the first gas volume value and the second gas volume value; The total amount of inhalation is determined according to the maximum air volume value, the digital air volume and the remaining air volume.

3. The method according to claim 2, characterized in that The determining the total amount of inhalation according to the maximum air volume value, the digital air volume and the remaining air volume includes: Determine a third popularity value according to the digital popularity and the second weight coefficient; Determining a minimum gas volume value between the maximum gas volume value and the third gas volume value; Determining a fourth popularity value according to the digital popularity and a third weight coefficient, wherein the sum of the second weight coefficient and the third weight coefficient is 1; The total inhaled air volume is determined according to the minimum air volume value, the fourth air volume value, and the remaining air volume.

4. The method according to claim 1, wherein The step of controlling the breathing and / or ventilation action of the digital human when broadcasting the first voice segment based on the relationship between the total amount of inhaled air and the digital human's breath volume includes: Determining a breathing control parameter according to the relationship between the total amount of inhaled air and the digital breath volume, the breathing control parameter comprising at least one of the following: a nostril expansion degree control parameter, a mouth expansion degree control parameter, a chest and abdomen expansion degree control parameter, and an inhalation volume control parameter; The breathing control parameters are input into the breathing model of the digital human, so that the breathing model controls the breathing and / or ventilation movements of the digital human according to the breathing control parameters.

5. The method according to claim 4, characterized in that Determining the breathing control parameter according to the relationship between the total amount of inspiration and the digital breath volume includes: Determine the inhalation pause time T according to the relationship between the total inhalation amount and the digital breath volume; determining the cumulative inspiratory volume at the first moment according to the inspiratory time t at the first moment and the pause time T; A breathing control parameter at the first moment is determined according to the accumulated inspiratory volume at the first moment.

6. The method according to claim 5, characterized in that The step of determining the inhalation pause time T according to the relationship between the total inhalation volume and the digital breath volume includes: When the total amount of inhaled air is less than a preset air volume value, the pause time T is determined to be a fixed sentence pause time; or When the total amount of inhalation is greater than or equal to the preset air volume value, the pause time T is determined according to the fixed sentence pause time, the digital air volume and the total amount of inhalation.

7. The method according to claim 5, characterized in that The determining of the cumulative inhaled volume at the first moment according to the inhaled time t and the pause time T at the first moment includes: When the ratio of the inhaled time t to the pause time T is less than a first threshold, the cumulative inhaled volume is determined according to a first functional relationship, wherein the instantaneous inhalation rate of the digital human is defined as a first rate in the first functional relationship; When the ratio of the inhaled time t to the pause time T is greater than or equal to a first threshold, the accumulated inhaled volume is determined according to a second functional relationship, wherein the instantaneous inhalation rate of the digital human is defined as a second rate in the second functional relationship; The first rate and the second rate are different.

8. The method according to claim 5, characterized in that The step of determining the breathing control parameter according to the accumulated inspiratory volume at the first moment includes: determining the chest and abdomen expansion degree control parameter according to the accumulated inspiratory volume; and / or, Determine the instantaneous inhalation speed at the first moment based on the accumulated inhalation volume at the first moment; and determine at least one of the nostril expansion degree control parameter, the mouth expansion degree control parameter, and the inhalation volume control parameter based on the instantaneous inhalation speed.

9. An electronic device comprising: A transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor; wherein the processor is configured to read the program in the memory to implement the steps of the digital human control method as claimed in any one of claims 1 to 8.

10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the digital human control method according to any one of claims 1 to 8.

Images