An interactive method and system for a virtual digital human

Abstract

The application discloses an interactive method and system of a virtual digital person, comprising the following steps: acquiring voice instruction data based on a virtual digital person scene, preprocessing the voice instruction data, extracting acoustic characteristics of the preprocessed voice instruction data, identifying instruction text of the voice instruction data, acquiring personal language habits and context information according to the instruction text, calculating personal characteristic values and matching an interactive database according to the acoustic characteristics, the personal language habits and the context information, establishing a virtual interactive model according to the interactive database, optimizing the virtual interactive model, inputting the instruction text into the optimized virtual interactive model, and outputting an interactive result. The method can not only improve the accuracy of interaction, but also has good interpretability and can be directly applied to an interactive system.

Description

Technical Field

[0001] This invention relates to the field of interaction, and more particularly to an interaction method and system for a virtual digital human. Background Technology

[0002] Virtual digital human technology is increasingly being applied in the field of interaction, helping interactive systems to analyze commands in a timely and efficient manner, enabling text analysis and dialogue. Currently, the characteristics of commands—such as their massive volume, diversity, and high information density—indicate numerous uncertainties in interaction methods, leading to significant uncertainty. Although some virtual digital human interaction methods and systems have been invented, they still cannot effectively solve the problem of uncertainty in interaction methods. Summary of the Invention

[0003] The purpose of this invention is to provide an interactive method for virtual digital humans.

[0004] To achieve the above objectives, the present invention is implemented according to the following technical solution:

[0005] This invention includes the following steps:

[0006] A acquires voice command data based on a virtual digital human scenario and preprocesses the voice command data;

[0007] B extracts the acoustic features of the preprocessed voice command data, identifies the command text of the voice command data, and obtains personal language habits and contextual information based on the command text;

[0008] C calculates personal characteristic values ​​based on the acoustic features, personal language habits, and contextual information, and matches them with the interactive database. The personal characteristic values ​​represent the degree of matching between the acoustic features, personal language habits, contextual information, and the historical database.

[0009] A method for calculating personal characteristic values ​​and matching them to an interactive database based on the acoustic features, personal language habits, and contextual information includes:

[0010] Calculate individual trait values:

[0011]

[0012] The feature set of the e-th interactive database data is A. e The acoustic feature set of the instruction text is B, with the weight of the acoustic features being ω1; the personal language habit set of the instruction text is C, with the weight of the personal habits being ω2; the contextual information set of the instruction text is D, with the weight of the contextual information being ω3; and the feature set is A. e The value is |A eThe numerical value of the instruction text is |ω1B+ω2C+ω3D|, and the interaction matching factor is ρ;

[0013] When the personal characteristic value is less than 0.5, the instruction text does not match the data in the interaction database. The data in the interaction database is replaced until the personal characteristic value is greater than or equal to 0.5. Then the e-th data in the interaction database is associated with the instruction text.

[0014] When the personal characteristic value is greater than or equal to 0.5, the e-th interactive database data is associated with the instruction text;

[0015] Replace the command text with the interactive database data until all command texts have been traversed;

[0016] D establishes a virtual interaction model based on the interaction database, optimizes the virtual interaction model, inputs the instruction text into the optimized virtual interaction model, and outputs the interaction result.

[0017] Furthermore, the preprocessing described in step A includes pre-emphasis, framing, windowing, endpoint detection, and denoising.

[0018] Furthermore, the method for extracting acoustic features from the preprocessed voice command data includes:

[0019] Perform a Fourier transform on the preprocessed voice command data to convert it to the frequency domain, and calculate the power spectral density.

[0020]

[0021] The power spectral density of the t-th sequence in the v-th frame is g(v,t), the length of each frame is M, the frequency domain representation of the t-th sequence in the v-th frame is h(v,t), the number of points in the Fourier transform is T, and the frequency response of the given filter is:

[0022]

[0023] The center frequency of the i-th filter is h(i), and the frequency response of the i-th filter in the v-th frame is W. i (v) Map the power spectral density to the Mel frequency scale using a triangular filter bank, and calculate the Mel frequency power spectral density:

[0024]

[0025] The number of triangular filters is I, and the Mel-frequency power spectral density of the i-th filter in the v-th frame is Q. V (v,i), performing a discrete cosine transform on the Mel frequency power spectral density yields the Mel frequency cepstral coefficients. The formula for calculating the Mel frequency cepstral coefficients is:

[0026]

[0027] Where the dimension of the Mel frequency cepstral coefficients is j, and the Mel frequency cepstral coefficient of the j-th dimension in the v-th frame is E(v,j), calculate the first difference of the Mel frequency cepstral coefficients:

[0028]

[0029] The frame range for the differential calculation is L, and the Mel frequency cepstral coefficient function of the j-th dimension of the (v+1)-th frame is E. j (v+1), the Mel frequency cepstral coefficient function of the j-th dimension of the (v-1)-th frame is E j (v-1), the first difference of the Mel frequency cepstral coefficients in the j-th dimension and the v-th frame is ΔE j (v), calculate the second difference of the Mel frequency cepstral coefficients:

[0030]

[0031] The second difference of the Mel frequency cepstral coefficients in the j-th dimension and v-th frame is divided into ΔΔE. j (v), the first difference of the Mel frequency cepstral coefficients of the j-th dimension and the (v-1)-th frame is ΔE j (v-1), the Mel frequency cepstral coefficient features are static features, the first-order difference of the Mel frequency cepstral coefficients and the second-order difference of the Mel frequency cepstral coefficients are dynamic features, and the output acoustic features are output.

[0032] Furthermore, the method for obtaining personal language habits based on the instruction text includes:

[0033] The instruction text is constructed into an instruction text database, and the support degree of transaction items in the database is calculated:

[0034]

[0035] The total number of transaction items is N, and the number of transactions containing a specific item m is C. m The support of transaction item m is s m Support statistics are performed on all items in the instruction text database, and items with support below the threshold are removed from the voice instruction data to obtain a prefix of length 1.

[0036] Find the projection database corresponding to the prefix. If the voice command data is empty, return recursively.

[0037] Count the support scores of each item in the corresponding projection database. If the support scores of all items are below the threshold, return recursively.

[0038] The support count item and the current prefix are merged to obtain a new prefix;

[0039] Increment the length by one, and use the prefix after merging individual items. Recursively process each item and return to determine whether the support count of each item is lower than the threshold.

[0040] Repeatedly mine prefixes whose length meets the support requirement until no new prefixes can be found, and output the prefixes according to personal language habits.

[0041] Furthermore, the method for obtaining the contextual information includes:

[0042] The instruction text is divided into six components: subject, predicate, object, attributive, adverbial, and complement; words or phrases are used as the basic units for dividing the components.

[0043] The structure of the instruction text is determined by the hierarchical arrangement of the six components. The semantics of the instruction text are then understood based on its structure, and the semantics of the instruction text are output as contextual information.

[0044] Furthermore, the method for establishing a virtual interaction model based on the interaction database includes:

[0045] The virtual interaction model includes a text preprocessing unit, a natural language understanding unit, a dialogue management unit, and a natural language unit. The instruction text is input into the text preprocessing unit to obtain preprocessed text, the preprocessed text is input into the natural language understanding unit to obtain understanding information, the understanding information is input into the dialogue management unit to obtain initial interaction text, and the initial interaction text is input into the natural language unit to output the interaction result.

[0046] The text preprocessing section is used to perform word segmentation, stop word removal, named entity recognition, word sense disambiguation, case conversion, and text conversion on the instruction text.

[0047] The Natural Language Understanding division is used to perform semantic parsing on preprocessed text, identify entities and intents in the text, and translate the semantic parsing into machine language.

[0048] The dialogue management division is responsible for generating system responses based on the understanding of the information status and the user's actions, deciding on the next steps, and updating the dialogue status.

[0049] The Natural Language Division is used to transform the initial interactive text into human-readable text.

[0050] Furthermore, the method for optimizing the virtual interaction model based on the interaction database includes:

[0051] The interactive database is randomly divided into a training dataset and a test dataset in an 8:2 ratio.

[0052] Set the hidden layer node to one, randomly give the set of interaction hyperparameter vectors, initialize the parameters, and group the training dataset for rolling validation;

[0053] Selecting groups of training datasets as training data to obtain the initial weights of the recurrent neural network;

[0054] Given the optimization expression of a nonlinear constrained quadratic function:

[0055]

[0056] The input variable and the cyclic weight of the previous time step of the l-th hidden layer node to the current time step of the j-th node are γ. lj The nth interaction result is x n The bias between the hidden layer and the output layer is z. 0 The input bias with respect to the j-th hidden layer is z. j The activation functions are f(·) and w(·), the time delay step is β, the number of hidden layer nodes is p, the number of candidate input variables i is q, the number of hidden layers is t, the time step is m, the interaction parameter is θ, and the i-th candidate input variable is a. i The i-th compression parameter is s i The b-th compression parameter is s b The hidden layer state at time m is y m (·), where the input weight of the i-th variable in the j-th hidden layer is e ij The weight between the j-th hidden layer node and the output is u. j ;

[0057] The compression coefficient is obtained by solving the nonlinear constraint quadratic function optimization expression, and the network weights are updated. The formula for calculating the updated network weights is as follows:

[0058] e′ ij =s i e ij

[0059] s′ b =s b γ lj

[0060] The updated input weight is e′ ij Updated recurrent weights s′ b .

[0061] Secondly, an interactive system for a digital virtual human includes:

[0062] Preprocessing module: acquires voice command data based on the virtual digital human scenario and preprocesses the voice command data;

[0063] Extraction module: Extracts the acoustic features of the preprocessed voice command data, identifies the command text of the voice command data, and obtains personal language habits and contextual information based on the command text;

[0064] Database matching module: Calculates personal characteristic values ​​based on the acoustic features, personal language habits, and contextual information, and matches them with the interactive database. The personal characteristic values ​​represent the degree of matching between the acoustic features, personal language habits, contextual information, and the historical database.

[0065] A method for calculating personal characteristic values ​​and matching them to an interactive database based on the acoustic features, personal language habits, and contextual information includes:

[0066] Calculate individual trait values:

[0067]

[0068] The feature set of the e-th interactive database data is A. e The acoustic feature set of the instruction text is B, with the weight of the acoustic features being ω1; the personal language habit set of the instruction text is C, with the weight of the personal habits being ω2; the contextual information set of the instruction text is D, with the weight of the contextual information being ω3; and the feature set is A. e The value is |A e The numerical value of the instruction text is |ω1B+ω2C+ω3D|, and the interaction matching factor is ρ;

[0069] When the personal characteristic value is less than 0.5, the instruction text does not match the data in the interaction database. The data in the interaction database is replaced until the personal characteristic value is greater than or equal to 0.5. Then the e-th data in the interaction database is associated with the instruction text.

[0070] When the personal characteristic value is greater than or equal to 0.5, the e-th interactive database data is associated with the instruction text;

[0071] Replace the command text with the interactive database data until all command texts have been traversed;

[0072] Interaction module: Establishes a virtual interaction model based on the interaction database, optimizes the virtual interaction model, inputs the instruction text into the optimized virtual interaction model, and outputs the interaction result.

[0073] The beneficial effects of this invention are:

[0074] This invention is an interactive method for virtual digital humans. Compared with existing technologies, this invention has the following technical advantages:

[0075] This invention improves the accuracy of virtual digital human interaction through preprocessing, feature extraction, database matching, model construction, model optimization, and virtual interaction steps. This enhances the precision of virtual digital human interaction and significantly saves resources and improves work efficiency. It enables automatic analysis and optimization of interaction, real-time feature extraction and database matching of voice command data, which is of great significance for virtual digital human interaction. It can adapt to the interaction of virtual digital humans with different standards and systems, and has a certain degree of universality. Attached Figure Description

[0076] Figure 1 This is a flowchart illustrating the steps of an interactive method for a virtual digital human according to the present invention. Detailed Implementation

[0077] The present invention will be further described below through specific embodiments. The illustrative embodiments and descriptions herein are used to explain the present invention, but are not intended to limit the present invention.

[0078] The present invention discloses an interactive method for a virtual digital human, comprising the following steps:

[0079] like Figure 1 As shown, this embodiment includes the following steps:

[0080] A acquires voice command data based on a virtual digital human scenario and preprocesses the voice command data;

[0081] In the actual evaluation, the user's voice command data, such as "Please help me find nearby restaurants and reserve a table for four," was used as input for analysis.

[0082] B extracts the acoustic features of the preprocessed voice command data, identifies the command text of the voice command data, and obtains personal language habits and contextual information based on the command text;

[0083] In the actual assessment, the acoustic features extracted from "Please help me find nearby restaurants and reserve a table for four" were as follows: syllable boundary pause duration averaged 1.47ms, intonation phrase boundary duration 195.72ms, fricative duration 54.59ms. The individual's language habits included glottal stops or syllables with glottal stops, a soft and delicate tone with rising and falling intonation, and a tendency towards colloquial expressions containing slang, colloquialisms, and verbal slang. Contextual information included:

[0084] Location information: The speaker is looking for nearby restaurants, indicating that they are likely looking for a place to eat near their current location;

[0085] Purpose and Intent: From the voice command, it can be seen that the speaker's intention is to find a restaurant and reserve a table for four;

[0086] Personnel information: The instruction clearly states that a table for four needs to be reserved, indicating that the speaker and three other people plan to dine together as a group of four.

[0087] Action type: This is a request or command type of context, where the speaker wants an action to be performed, namely, finding a restaurant and making a table reservation;

[0088] C calculates personal characteristic values ​​based on the acoustic features, personal language habits, and contextual information, and matches them with the interactive database. The personal characteristic values ​​represent the degree of matching between the acoustic features, personal language habits, contextual information, and the historical database.

[0089] A method for calculating personal characteristic values ​​and matching them to an interactive database based on the acoustic features, personal language habits, and contextual information includes:

[0090] Calculate individual trait values:

[0091]

[0092] The feature set of the e-th interactive database data is A. e The acoustic feature set of the instruction text is B, with the weight of the acoustic features being ω1; the personal language habit set of the instruction text is C, with the weight of the personal habits being ω2; the contextual information set of the instruction text is D, with the weight of the contextual information being ω3; and the feature set is A. e The value is |A e The numerical value of the instruction text is |ω1B+ω2C+ω3D|, and the interaction matching factor is ρ;

[0093] When the personal characteristic value is less than 0.5, the instruction text does not match the data in the interaction database. The data in the interaction database is replaced until the personal characteristic value is greater than or equal to 0.5. Then the e-th data in the interaction database is associated with the instruction text.

[0094] When the personal characteristic value is greater than or equal to 0.5, the e-th interactive database data is associated with the instruction text;

[0095] Replace the command text with the interactive database data until all command texts have been traversed;

[0096] In the actual evaluation, the weight of acoustic features was 0.32, the weight of personal language habits was 0.4, the weight of contextual information was 0.28, the personal characteristic value of database 1 was 0.34, the personal characteristic value of database 2 was 0.17, the personal characteristic value of database 3 was 0.72, the personal characteristic value of database 4 was 0.41, and the matching interaction database was database 3.

[0097] D establishes a virtual interaction model based on the interaction database, optimizes the virtual interaction model, inputs the instruction text into the optimized virtual interaction model, and outputs the interaction result.

[0098] In the actual evaluation, the interaction result was: "Okay, we have filtered the 567 yuan four-person meal at XXX restaurant for you, with a walking distance of 205m."

[0099] In this embodiment, the preprocessing in step A includes pre-emphasis, framing, windowing, endpoint detection, and denoising.

[0100] In this embodiment, the method for extracting the acoustic features of the preprocessed voice command data includes:

[0101] Perform a Fourier transform on the preprocessed voice command data to convert it to the frequency domain, and calculate the power spectral density.

[0102]

[0103] The power spectral density of the t-th sequence in the v-th frame is g(v,t), the length of each frame is M, the frequency domain representation of the t-th sequence in the v-th frame is h(v,t), the number of points in the Fourier transform is T, and the frequency response of the given filter is:

[0104]

[0105] The center frequency of the i-th filter is h(i), and the frequency response of the i-th filter in the v-th frame is W. i (v) Map the power spectral density to the Mel frequency scale using a triangular filter bank, and calculate the Mel frequency power spectral density:

[0106]

[0107] The number of triangular filters is I, and the Mel-frequency power spectral density of the i-th filter in the v-th frame is Q. V (v,i), performing a discrete cosine transform on the Mel frequency power spectral density yields the Mel frequency cepstral coefficients. The formula for calculating the Mel frequency cepstral coefficients is:

[0108]

[0109] Where the dimension of the Mel frequency cepstral coefficients is j, and the Mel frequency cepstral coefficient of the j-th dimension in the v-th frame is E(v,j), calculate the first difference of the Mel frequency cepstral coefficients:

[0110]

[0111] The frame range for the differential calculation is L, and the Mel frequency cepstral coefficient function of the j-th dimension of the (v+1)-th frame is E.j (v+1), the Mel frequency cepstral coefficient function of the j-th dimension of the v-1th frame is E j (v-1), the first difference of the Mel frequency cepstral coefficients in the j-th dimension and the v-th frame is ΔE j (v), calculate the second difference of the Mel frequency cepstral coefficients:

[0112]

[0113] The second difference of the Mel frequency cepstral coefficients in the j-th dimension and v-th frame is divided into ΔΔE. j (v), the first difference of the Mel frequency cepstral coefficients of the j-th dimension and the (v-1)-th frame is ΔE j (v-1), the Mel frequency cepstral coefficient features are static features, the first-order difference of the Mel frequency cepstral coefficients and the second-order difference of the Mel frequency cepstral coefficients are dynamic features, and the output acoustic features are output.

[0114] In this embodiment, the method for obtaining personal language habits based on the instruction text includes:

[0115] The instruction text is constructed into an instruction text database, and the support degree of transaction items in the database is calculated:

[0116]

[0117] The total number of transaction items is N, and the number of transactions containing a specific item m is C. m The support of transaction item m is s m Support statistics are performed on all items in the instruction text database, and items with support below the threshold are removed from the voice instruction data to obtain a prefix of length 1.

[0118] Find the projection database corresponding to the prefix. If the voice command data is empty, return recursively.

[0119] Count the support scores of each item in the corresponding projection database. If the support scores of all items are below the threshold, return recursively.

[0120] The support count item and the current prefix are merged to obtain a new prefix;

[0121] Increment the length by one, and use the prefix after merging individual items. Recursively process each item and return to determine whether the support count of each item is lower than the threshold.

[0122] Repeatedly mine prefixes whose length meets the support requirement until no new prefixes can be found, and output the prefixes according to personal language habits.

[0123] In this embodiment, the method for obtaining the contextual information includes:

[0124] The instruction text is divided into six components: subject, predicate, object, attributive, adverbial, and complement; words or phrases are used as the basic units for dividing the components.

[0125] The structure of the instruction text is determined by the hierarchical arrangement of the six components. The semantics of the instruction text are then understood based on its structure, and the semantics of the instruction text are output as contextual information.

[0126] In this embodiment, the method for establishing a virtual interaction model based on the interaction database includes:

[0127] The virtual interaction model includes a text preprocessing division, a natural language understanding division, a dialogue management division, and a natural language division. The instruction text is input into the text preprocessing division to obtain preprocessed text, the preprocessed text is input into the natural language understanding division to obtain understanding information, the understanding information is input into the dialogue management division to obtain initial interaction text, and the initial interaction text is input into the natural language division to output the interaction result.

[0128] The text preprocessing section is used to perform word segmentation, stop word removal, named entity recognition, word sense disambiguation, case conversion, and text conversion on the instruction text.

[0129] The Natural Language Understanding division is used to perform semantic parsing on preprocessed text, identify entities and intents in the text, and translate the semantic parsing into machine language.

[0130] The dialogue management division is responsible for generating system responses based on the understanding of the information status and the user's actions, deciding on the next steps, and updating the dialogue status.

[0131] The Natural Language Division is used to transform the initial interactive text into human-readable text.

[0132] In this embodiment, the method for optimizing the virtual interaction model based on the interaction database includes:

[0133] The interactive database is randomly divided into a training dataset and a test dataset in an 8:2 ratio.

[0134] Set the hidden layer node to one, randomly give the set of interaction hyperparameter vectors, initialize the parameters, and group the training dataset for rolling validation;

[0135] Selecting groups of training datasets as training data to obtain the initial weights of the recurrent neural network;

[0136] Given the optimization expression of a nonlinear constrained quadratic function:

[0137]

[0138] The input variable and the cyclic weight of the previous time step of the l-th hidden layer node to the current time step of the j-th node are γ. lj The nth interaction result is x n The bias between the hidden layer and the output layer is z. 0 The input bias with respect to the j-th hidden layer is z. j The activation functions are f(·) and w(·), the time delay step is β, the number of hidden layer nodes is p, the number of candidate input variables i is q, the number of hidden layers is t, the time step is m, the interaction parameter is θ, and the i-th candidate input variable is a. i The i-th compression parameter is s i The b-th compression parameter is s b The hidden layer state at time m is y m (·), where the input weight of the i-th variable in the j-th hidden layer is e ij The weight between the j-th hidden layer node and the output is u. j ;

[0139] The compression coefficient is obtained by solving the nonlinear constraint quadratic function optimization expression, and the network weights are updated. The formula for calculating the updated network weights is as follows:

[0140] e′ ij =s i e ij

[0141] s′ b =s b γ lj

[0142] The updated input weight is e′ ij Updated recurrent weights s′ b .

[0143] Secondly, an interactive system for a digital virtual human includes:

[0144] Preprocessing module: acquires voice command data based on the virtual digital human scenario and preprocesses the voice command data;

[0145] Extraction module: Extracts the acoustic features of the preprocessed voice command data, identifies the command text of the voice command data, and obtains personal language habits and contextual information based on the command text;

[0146] Database matching module: Calculates personal characteristic values ​​based on the acoustic features, personal language habits, and contextual information, and matches them with the interactive database. The personal characteristic values ​​represent the degree of matching between the acoustic features, personal language habits, contextual information, and the historical database.

[0147] A method for calculating personal characteristic values ​​and matching them to an interactive database based on the acoustic features, personal language habits, and contextual information includes:

[0148] Calculate individual trait values:

[0149]

[0150] The feature set of the e-th interactive database data is A. e The acoustic feature set of the instruction text is B, with the weight of the acoustic features being ω1; the personal language habit set of the instruction text is C, with the weight of the personal habits being ω2; the contextual information set of the instruction text is D, with the weight of the contextual information being ω3; and the feature set is A. e The value is |A e The numerical value of the instruction text is |ω1B+ω2C+ω3D|, and the interaction matching factor is ρ;

[0151] When the personal characteristic value is less than 0.5, the instruction text does not match the data in the interaction database. The data in the interaction database is replaced until the personal characteristic value is greater than or equal to 0.5. Then the e-th data in the interaction database is associated with the instruction text.

[0152] When the personal characteristic value is greater than or equal to 0.5, the e-th interactive database data is associated with the instruction text;

[0153] Replace the command text with the interactive database data until all command texts have been traversed;

[0154] Interaction module: Establishes a virtual interaction model based on the interaction database, optimizes the virtual interaction model, inputs the instruction text into the optimized virtual interaction model, and outputs the interaction result.

[0155] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An interaction method for a virtual digital human, characterized in that, Includes the following steps: A acquires voice command data based on a virtual digital human scenario and preprocesses the voice command data; B extracts the acoustic features of the preprocessed voice command data, identifies the command text of the voice command data, and obtains personal language habits and contextual information based on the command text; C calculates personal characteristic values ​​based on the acoustic features, personal language habits, and contextual information, and matches them with the interactive database. The personal characteristic values ​​represent the degree of matching between the acoustic features, personal language habits, contextual information, and historical database. A method for calculating personal characteristic values ​​and matching them to an interactive database based on the acoustic features, personal language habits, and contextual information includes: Calculate individual trait values: ; The feature set of the e-th interactive database data is as follows: The acoustic feature set of the instruction text is B, and the weights of the acoustic features are... Let C be the set of individual language habits in the instruction text, and let the weight of each individual habit be... The set of contextual information in the instruction text is D, and the weight of the contextual information is . Feature set The value is The numerical value of the instruction text is The interaction matching factor is ; When the personal characteristic value is less than 0.5, the instruction text does not match the data in the interaction database. The data in the interaction database is replaced until the personal characteristic value is greater than or equal to 0.

5. Then the e-th data in the interaction database is associated with the instruction text. When the personal characteristic value is greater than or equal to 0.5, the e-th interactive database data is associated with the instruction text; Replace the command text with the interactive database data until all command texts have been traversed; D establishes a virtual interaction model based on the interaction database, optimizes the virtual interaction model, inputs the instruction text into the optimized virtual interaction model, and outputs the interaction result.

2. The interaction method for a virtual digital human according to claim 1, characterized in that, The preprocessing described in step A includes pre-emphasis, framing, windowing, endpoint detection, and denoising.

3. The interaction method for a virtual digital human according to claim 1, characterized in that, A method for extracting acoustic features from the preprocessed voice command data includes: Perform a Fourier transform on the preprocessed voice command data to convert it to the frequency domain, and calculate the power spectral density. ; The power spectral density of the t-th sequence in the v-th frame is: Each frame has a length of M, and the frequency domain representation of the t-th sequence in the v-th frame is as follows: Given a filter with T points in its Fourier transform and the following frequency response: ; The center frequency of the i-th filter is The frequency response of the i-th filter in the v-th frame is The power spectral density is mapped to the Mel frequency scale using a triangular filter bank, and the Mel frequency power spectral density is calculated. ; The number of triangular filters is I, and the Mel-frequency power spectral density of the i-th filter in the v-th frame is... The Mel frequency cepstral coefficients are obtained by performing a discrete cosine transform on the Mel frequency power spectral density. The formula for calculating the Mel frequency cepstral coefficients is as follows: ; The dimension of the Mel frequency cepstral coefficients is j, and the Mel frequency cepstral coefficients of the j-th dimension of the v-th frame are: Calculate the first-order difference of the Mel frequency cepstral coefficients: ; The frame range for the difference calculation is L, and the Mel frequency cepstral coefficient function of the j-th dimension of the (v+1)-th frame is: The Mel frequency cepstral coefficient function of the j-th dimension of the v-1-th frame is: The first-order difference of the Mel frequency cepstral coefficients of the j-th dimension and the v-th frame is divided into Calculate the second difference of the Mel frequency cepstral coefficients: ; The second difference of the Mel frequency cepstral coefficients in the j-th dimension and the v-th frame is divided into... The first-order difference of the Mel frequency cepstral coefficients in the j-th dimension and the (v-1)-th frame is divided into The Mel frequency cepstral coefficients are static features, while the first-order difference and second-order difference of the Mel frequency cepstral coefficients are dynamic features, resulting in output acoustic features.

4. The interaction method for a virtual digital human according to claim 1, characterized in that, A method for obtaining personal language habits based on the instruction text includes: The instruction text is constructed into an instruction text database, and the support degree of transaction items in the database is calculated: ; The total number of transaction items is N, and the number of transactions containing a specific item m is... The support of transaction item m is Support statistics are performed on all items in the instruction text database, and items with support below the threshold are removed from the voice instruction data to obtain a prefix of length 1. Find the projection database corresponding to the prefix. If the voice command data is empty, return recursively. Count the support scores of each item in the corresponding projection database. If the support scores of all items are below the threshold, return recursively. The support count item and the current prefix are merged to obtain a new prefix; Increment the length by one, and use the prefix after merging individual items. Recursively process each item and return to determine whether the support count of each item is lower than the threshold. Repeatedly mine prefixes whose length meets the support requirement until no new prefixes can be found, and output the prefixes according to personal language habits.

5. The interaction method for a virtual digital human according to claim 1, characterized in that, The method for obtaining the contextual information includes: The instruction text is divided into six components: subject, predicate, object, attributive, adverbial, and complement; words or phrases are used as the basic units for dividing the components. The structure of the instruction text is determined by the hierarchical arrangement of the six components. The semantics of the instruction text are then understood based on its structure, and the semantics of the instruction text are output as contextual information.

6. The interaction method for a virtual digital human according to claim 1, characterized in that, The method for establishing a virtual interaction model based on the interaction database includes: The virtual interaction model includes a text preprocessing division, a natural language understanding division, a dialogue management division, and a natural language division. The instruction text is input into the text preprocessing division to obtain preprocessed text, the preprocessed text is input into the natural language understanding division to obtain understanding information, the understanding information is input into the dialogue management division to obtain initial interaction text, and the initial interaction text is input into the natural language division to output the interaction result. The text preprocessing section is used to perform word segmentation, stop word removal, named entity recognition, word sense disambiguation, case conversion, and text conversion on the instruction text. The Natural Language Understanding division is used to perform semantic parsing on preprocessed text, identify entities and intents in the text, and translate the semantic parsing into machine language. The dialogue management division is responsible for generating system responses based on the understanding of the information status and the user's actions, deciding on the next steps, and updating the dialogue status. The Natural Language Division is used to transform the initial interactive text into human-readable text.

7. The interaction method for a virtual digital human according to claim 1, characterized in that, A method for optimizing the virtual interaction model based on the interaction database includes: The interactive database is randomly divided into a training dataset and a test dataset in an 8:2 ratio. Set the hidden layer node to one, randomly give the set of interaction hyperparameter vectors, initialize the parameters, and group the training dataset for rolling validation; Selecting groups of training datasets as training data to obtain the initial weights of the recurrent neural network; Given the optimization expression of a nonlinear constrained quadratic function: ; s.t. ; ; The input variable and the cyclic weight of the previous time step of the l-th hidden layer node with respect to the current time step of the j-th node are: The nth interaction result is The bias between the hidden layer and the output layer is The input bias with respect to the j-th hidden layer is The activation function is and The time delay step is The number of hidden layer nodes is p, the number of candidate input variables i is q, the number of hidden layers is t, the time step is m, and the interaction parameters are... The i-th candidate input variable is The i-th compression parameter is The b-th compression parameter is The state of the hidden layer at time m is The input weights of the i-th variable and the j-th hidden layer are: The weight between the j-th hidden layer node and the output is ; The compression coefficient is obtained by solving the nonlinear constraint quadratic function optimization expression, and the network weights are updated. The formula for calculating the updated network weights is as follows: ; ; The updated input weights are: Updated loop weights .

8. An interactive system for a virtual digital human, characterized in that, include: Preprocessing module: acquires voice command data based on the virtual digital human scenario and preprocesses the voice command data; Extraction module: Extracts the acoustic features of the preprocessed voice command data, identifies the command text of the voice command data, and obtains personal language habits and contextual information based on the command text; Database matching module: Calculates personal characteristic values ​​based on the acoustic features, personal language habits, and contextual information, and matches them with the interactive database. The personal characteristic values ​​represent the degree of matching between the acoustic features, personal language habits, contextual information, and historical database. A method for calculating personal characteristic values ​​and matching them to an interactive database based on the acoustic features, personal language habits, and contextual information includes: Calculate individual trait values: ; The feature set of the e-th interactive database data is as follows: The acoustic feature set of the instruction text is B, and the weights of the acoustic features are... Let C be the set of individual language habits in the instruction text, and let the weight of each individual habit be... The set of contextual information in the instruction text is D, and the weight of the contextual information is . Feature set The value is The numerical value of the instruction text is The interaction matching factor is ; When the personal characteristic value is less than 0.5, the instruction text does not match the data in the interaction database. The data in the interaction database is replaced until the personal characteristic value is greater than or equal to 0.

5. Then the e-th data in the interaction database is associated with the instruction text. When the personal characteristic value is greater than or equal to 0.5, the e-th interactive database data is associated with the instruction text; Replace the command text with the interactive database data until all command texts have been traversed; Interaction module: Establishes a virtual interaction model based on the interaction database, optimizes the virtual interaction model, inputs the instruction text into the optimized virtual interaction model, and outputs the interaction result.

Images